AddRelativeTorque and gameobject's angularVelocity

I've been experimenting around with some of the physics in Unity and something seems weird to me with the AddRelativeTorque function. I have the following code in order to slow and eventually stop a gameobject's spinning.

         if ( ship.angularVelocity != Vector3.zero )
         {
             ship.AddRelativeTorque( ship.angularVelocity * -1 );
         }

Elsewhere I have this line to apply a spin to it using input from a controller or keyboard.

     ship.AddRelativeTorque( Vector3.up * Input.GetAxis( "Rotate" ) / 2 );

I don't fully understand how the physics behind AddTorque work, but from what I understand, this should apply a force opposite to the current velocity, thus slowing down and stopping the object. It does work, but if I apply enough other forces on the object, then it'll start spinning uncontrollably.

I know how to achieve what I'm trying to do in a number of different ways that I know work. I just want to know what is happening and why it's happening. I assume it has something to do with how Unity handles forces and other physics, but I don't fully understand what's going on.

In case there's something in the rest of the code, here is the full set of functions. It happens most often when the explosion force is activated several times in a row, but the strange spinning will still occur even if I'm just navigating around the world and not colliding with anything.

     void moveShip ()
     {
         if ( !collided )
         {
             ship.AddRelativeForce( Vector3.forward * Input.GetAxis( "Vertical" ) );
 
             ship.AddRelativeForce( Vector3.right * Input.GetAxis( "Horizontal" ) );
 
             ship.AddRelativeForce( Vector3.up * -Input.GetAxis( "Up" ) );
 
             ship.AddRelativeTorque( Vector3.up * Input.GetAxis( "Rotate" ) / 2 );
         }
 
         if ( ship.velocity != Vector3.zero )
         {
             ship.AddRelativeForce( -ship.velocity.x / 2, - ship.velocity.y / 2, -ship.velocity.z / 2 );
         }
 
         if ( ship.angularVelocity != Vector3.zero )
         {
             ship.AddRelativeTorque( ship.angularVelocity * -1 );
         }
 
     }
 
     void OnCollision( Collision obj )
     {
         if ( obj.gameObject.CompareTag( "Cube" ) )
         {
             ship.AddExplosionForce( 1, obj.transform.position, 1 );
 
             collided = true;
         }
     }
 
     void OnCollisionExit( Collision obj )
     {
         if ( obj.gameObject.CompareTag( "Cube" ) )
         {
             collided = false;
         }
     }