Rigidbodies move slightly into each other

I am working on a Platformer game and I need very direct controls. I figured that transform.translate would be the way to go, and it works like a charm, but if my character collides with another rigidbody, then they move slightly into each other. The character gets pushed out when I stop moving forward, but I can't prevent them from moving into each other.

What am I doing wrong?