Box Collider bounce causing rigidbody to change rotation

Hey Unity Wizards, I'm new to Unity Colliders so maybe someone could explain the behavior I'm getting so I can find a fix for it.

My Scene

Here is a screenshot of my scene, in it's finished form I would like for it to be a 2.5D top down space game similar to asteroids.

As you can see I have a ship, a plane for the background and box boundaries around the level.

My game in motion with the boundary drawn in

My ship keeps it's rotation as long as it's within the boundary, with the camera following it. It moves around nicely.

But when the rigidbody ship hits the box collider boundary, it appears to almost be bouncing off the box collider, and then losing it's rotation angle and veering off in the y-axis (camera is top down)

Ship: "Bye suckers!"

So I guess the ideal behavior is for my ship to just bump up against the wall and stay there, I'm not sure if the best way to do this is to rely on the collision to stop it or for the ship to ignore translate calls while it's pressing against the boundary.

Player Source and ship rigidbody properties

Can anyone point me in the right direction with these colliders? thanks.