How to fix player bumping into the wall?

It feels like you are moving the square with .Translate() or .position + = moveVector which is not very correct. If your player object has a Rigidbody or Rigidbody2D component, then there are several ways to move the player:

First, we get the Rigidbody2D component:

 Rigidbody2D rb2D = GetComponent<Rigidbody2D>();

1) With the .velocity destination:

 rb2D.velocity = moveVector;

But you should be very careful when changing the velocity. You have to take into account that you need to keep the velocity.y value in order for the player to fall correctly.

2) By adding force ( .AddForce()):

 rb2D.AddForce(moveVector);

But here, too, you should be careful. In this case, the player will be able to accelerate infinitely, so you need to limit the horizontal speed:

 // declare and assign variable |maxSpeed|
 var xSpeed = rb2D.velocity.x;
 xSpeed = Mathf.Clamp(xSpeed, -maxSpeed, maxSpeed); 
 rb2D.velocity = new Vector2(xSpeed, rb2D.velocity.y);

3) With .MovePosition():

 rb2D.MovePosition(transform.position + moveVector);

This method is probably more suitable for you than others.