Weird collision 2D problem?

So I have a top down 2D scene I am playing around with and I have a player object that has a BoxCollider2D and Rigidbody2D components and a tile for a wall that also have a BoxCollider2D component. Neither of the BoxCollider2D component are triggers. I have a straight line of these wall tiles and when I move my player next to them, it stops as expected. The unexpected behavior I am seeing if that once I hit the wall and try to walk along the side of it, sometime the collider of the next wall tile will stop my player from moving forward. This does not make sense to me because the wall are in a straight line, why would the collider of the wall I am next to be different than the next wall.

Another weird thing is that I seem to be able to get rid of the issue by change the BoxCollder2D size of the player to 0.99. Not sure why this would fix it as I would expect the collision of the colliders to be the same regradless of the size of them.

What would be causing this issue? Is it expected that I would need to change the size of the BoxCollider2D to get this to seemingly work properly?