Grid-based building system not working

Hey.

I'm making a survival game and I am trying to build a building system similar to Rust.

This is my code:

 using System.Collections;
 using System.Collections.Generic;
 using UnityEngine;
 
 public class SnapToGrid : MonoBehaviour {
 
     public GameObject buildingPrefab;
 
     public float gridDistance = 2f;
 
     float gridSize;
 
     Vector3 gridPosition;
     Vector3 worldPosition;
 
     void Start() {
         worldPosition = transform.position;
 
         gridSize = 1f / gridDistance;
     }
 
     void Update() {
 
         gridSize = 1f / gridDistance;
 
         Ray ray = Camera.main.ScreenPointToRay(new Vector3(Input.mousePosition.x, Input.mousePosition.y, 0f));
 
         RaycastHit hit;
 
         Physics.Raycast(ray, out hit);
 
         worldPosition = hit.point;
 
         //Vector3 distanceVec = hit.point - Camera.main.transform.position;
         //worldPosition -= distanceVec.normalized * (distanceVec.magnitude - 1f);
 
         gridPosition = new Vector3(
         Mathf.Round(worldPosition.x * gridSize) / gridSize,
         Mathf.Round(worldPosition.y * gridSize) / gridSize,
         Mathf.Round(worldPosition.z * gridSize) / gridSize);
 
         transform.position = gridPosition;
 
         if(Input.GetMouseButtonDown(0)) {
             Instantiate(buildingPrefab, gridPosition, Quaternion.identity);
         }
     }
 }

There is a preview where the raycast from the camera comes in contact with a collider, and whenever you click it will instantiate the building in the place where the preview is.

HOWEVER... The problem I currently have is whever you try to place a building on the side of another building, it clips through and goes inside that building.

This makes it impossible to place one building on the side of another, and it also makes it so you can stack multiple buildings in the exact same position. These are both things I do not want.

Do you guys know of any way to make the preview on the side of the placed building instead of inside it? Thanks, Rugbug