how to rotate a Rigidbody to a target rotation using Rigidbody.AddTorque()

In my game the player can pick up objects using physics like with the gravity gun in half-life or portal. I want the objects to rotate with the player. I'm currently using Quaternion.Slerp() and RigidbodyConstraints to rotate a Rigidbody to a target rotation

 rb.transform.rotation = Quaternion.Slerp(rb.transform.rotation, targetRotation, smoothRotation);

but the Rigidbody can be rotated through other objects so I want to use Rigidbody.AddTorque() to rotate it using the physics engine