Rigidbody MovePosition clipping

When I use Rigidbody.MovePosition and the object being moved hits another Rigidbody, the object will overlap the other slightly until the MovePosition is released (eg, moving the player, if a direction is held "into" a wall, the player will clip into the wall very slightly, then slide back out when the direction is released.

I am using FixedUpdate, Box Colliders and non-kinematic rigidbodies.

 public class MoveController : MonoBehaviour {
 
     public float walkSpeed = 4f;
     public float acceleration = 5f;
     public float deceleration = 20f;
 
     Rigidbody rb;
     float currentSpeed;
     Vector3 movement;
 
     // Use this for initialization
     void Start () {
         rb = GetComponent<Rigidbody>();
     }
     
     // Update is called once per frame
     void FixedUpdate () {
         float moveHorizontal = Input.GetAxisRaw("Horizontal");
         float moveVertical = Input.GetAxisRaw("Vertical");
 
         if (moveHorizontal == 0f && moveVertical == 0f) {
             currentSpeed = Mathf.Lerp(currentSpeed, 0f, deceleration * Time.deltaTime);
         }
         else {
             currentSpeed = Mathf.Lerp(currentSpeed, walkSpeed, acceleration * Time.deltaTime);
         }
 
         movement.Set(moveHorizontal, 0.0f, moveVertical);
         movement = movement.normalized * currentSpeed * Time.deltaTime;
 
         rb.MovePosition(rb.position + movement);
     }
 }

This gif demonstrates the clipping behaviour:

How do I prevent this? It particularly causes problems when an object in mid-air collides with a wall, it will "stick" to it and stop falling until there is no horizontal push being applied.