Remove rigidbody object

Hi all, im making a game in which you can swipe an object to the left or right to remove it from a stack, I am using the drag rigid body script and have modified it to remove the object from the scene once it gets a certain distance from the main stack after its swiped there. However I am having some trouble, firstly the object itself is removed but the scene still has the drag rigid body object in the scene that is created by the script. Also having all these rigid bodys on the objects in the stack seems to cause bouncing issues.

Does anyone have a better idea of how to do this? or any help on solving these issues would be appreciated.

Thanks