Collision issues with changing direct velocity

I have a player object that needs to change velocity directly towards where he is heading with this code (inside the function void FixedUpdate())

 // The player ships moves directly to left or right depending on ControllerScript.playerRotation which is -1 or 1 or else 0.
 rigidbody.velocity = Quaternion.AngleAxis (ControllerScript.playerRotation * turnSpeed, Vector3.forward) * rigidbody.velocity;
 transform.rotation = Quaternion.LookRotation (rigidbody.velocity, Vector3.forward);

This code is working great, however, since the rigidbody slows down with gravity in my 2.5d platformer, at some moment it should just stop the object. But when the ship is moving to slow it starts spinning in all kinds of directions. The normalized velocity changes every frame from -1 to 1 in the y for some reason. The object should be stable when the magnitude of the velocity is 0. But doing that hard coded like if (rigidbody.magnitude < 0.1) seems a bit odd. Also because this could stop the object in mid air when turning around.