Need help with configurable joint (VR Pickupable like shopping bag)

So I'm making a pickupable system in VR, and using joints because the object you picked up still needs to be able to interact with physics. Now, what I'm trying to make the pickupable rotate only on a certain axis. For example: consider a shopping bag. When you hold this bag you can rotate it on the Y axis, but not on the Z or X axis. This is exactly what I want to achieve, but I need help configuring the configurable joint :)

What I've tried: I set the Angular X and Z motion to free, while keeping the Angular Y motion on locked. This gave kind of the desired result, but the problem was that the object kept rotating when it recieved angular force. What it should do is: rotate when colliding so that it doesn't clip through other objects, but snap back to its original rotation when it can.

Much like the swords in this video: https://www.youtube.com/watch?v=JNhEMRz91ag

Does anyone know how i can do this, and how i need to configure the joint?