Unity 2D - Rigidbody2D Velocity sudden freeze

I am using code to dynamically load my levels from images and I have double checked that everything is snapped with no holes in the collider structure.
However for some odd reason when I move my character using the Unity's 2D Platform Controller the character just freezes and I have to unfreeze it either by going back or by jumping.
I wasn't able to find any solutions online so I'm asking here.