Player sprite clipping through game boundary in 2d scene

Hello,

I'm a newb learning Unity and working on a Geometry Wars clone project. I'm having an issue where my player ship is clipping through the game boundary if I continually attempt to pass through it moving back and forth. Here is a video of the issue in action:

Video of issue

My boundary has a Box Collider 2D attached to it and my player ship has a Rigidbody 2D. My player movement function is quite basic:

 private void movePlayer()
 {
     Vector3 movement = new Vector3(inputDeltaX.Value, inputDeltaY.Value, 0) * speed.Value * Time.deltaTime;
     Vector3 newPosition = transform.position + movement;
     rigidBody.MovePosition(newPosition);
 }

Boundary Box Collider 2D settings:

Player ship Rigidbody 2D:

The player_collision material has a friction and bounciness value of 0.

My theory about the issue is that because I'm manually setting the transform of my player ship on update, the updated position is within the bounds of the boundary. The physics system tries to process this as a collision and sometimes ends up pushing the player out through the other side of the boundary.

Can anyone confirm if this is what could be happening, or is there anything obvious I'm doing incorrectly here? Thanks for your time.