Rigidbody Sliding UP Incline

Hello everyone, I'm noticing a very strange behavior with a rigidbody, it is actually siding up slopes. I've noticed this behavior after I added a hinge joint to some teeter-totter like objects in a game scene.

Video of issue: https://www.youtube.com/watch?v=ykW1Twdgwao&feature=youtu.be

Without the hinge joint, everything seems to work fine (but I need that hinge joint because as far as I've seen, the rigidbody constraints are only in world space, but the hinge joint works in local space).

If anyone has an idea as to why this is happening, I would love to know about it.