My Click and drag object moves slowly after I drop it

I have a game where you must click and drag a box and put them on top of each other and the boxes should be able to fall forward and backwards if you take them to far on the edgem so I dont want to put on constraints on my Rigidbody, but when I have a box on top of the other and the top starts going forward on the box and then after a few seconds go too far on the edge and fall... Is there anything I should set up at the Rigidbody? I have tried all 3 types of Collision Detection, my angular drag and Drag is 0, Mass 1, not kinematic or Interpolate, but same result... I have also added a Max Friction to my box collider