Move object with mouse without passing through colliders

Hi,

I have been working on something for along time now and cant seem to understand what goes wrong. Basically i place an object on a plane and if i want to place a second object i can just place it inside the other object. Both objects have colliders and rigidbody's and ar frozen in position and rotation.

I use transform.position to make the object follow the mouse (raycast hit position), i believe this is the problem why it ignores the colliders. I have tried alot of weird solution since the problem isnt there when i unfreeze positions ( it will push the other away, not good).

My question comes down to this i need a way to make the object follow my mouse (raycast) but still use collisions to not pass through each other. I cant seem to get rigidbody addforce to work because it just shoots in a weird direction....

any help would be appreciated.