Character controller sent flying after collision with a moving GameObject

In the game I am creating, an enemy with a collider chases the character controller around using NavMeshAgent. The problem is, when the character controller is moving towards the enemy and the enemy is moving towards it, when they collide the character controller is sent flying into the air. I can't get rid of the collider on the enemy, and I have checked that there is no collider on the player.

Any help/explanation for why this is happening?