Prevent the rotation

I have a character which has some animations

and I am using for rotation

 rigidbody.AddTorque(new Vector3(0, rotationFactor * Input.GetAxis("Horizontal"), 0));

and for forward movement

 rigidbody.AddForce(rigidbody.transform.TransformDirection(Vector3.forward * Input.GetAxis("Vertical") * speedFactor));

I have two animations on it, one for when it is idle and one for when it is running, the problem is that these animation cause a slight rotation change of the character, even in the idle case. I want to have rotation only because of the AddTorque, is it any way to prevent the rotation that is caused by the animation?

I found out that the problem is not the animation, the problem probably is in the function that is moving the character, when I am trying to move it forward it is also rotating on the left side, I can not understand why. Any idea?