Help with rigidbody slowing (inertia dampening)

I am making a space game where the player controls a spaceship that moves according to newton's laws of motion. The spacecraft can translate forward/back, up/down and left/right. It can also pitch, yaw and roll.

Code for this:

     Rigidbody rb;
     
     public float thrustForward;
     public float thrustBack;
     public float thrustUp;
     public float thrustDown;
     public float thrustLeft;
     public float thrustRight;
     public float thrustPitch;
     public float thrustYaw;
     public float thrustRoll;
     public float mouseSteerSensitivity;
     
     float xThrust;
     float yThrust;
     float zThrust;
     float xRot;
     float yRot;
     float zRot;
     float mouseXSteer;
     float mouseYSteer;
     
     bool inertiaDampeners;

 //reset thrusters for this frame
         zThrust = 0;
         yThrust = 0;
         xThrust = 0;
         xRot = 0;
         yRot = 0;
         zRot = 0;
         
         //get the active thrusters
         if (Input.GetKey(KeyCode.W)) {zThrust = thrustForward;}
         else if (Input.GetKey(KeyCode.S)) {zThrust = -thrustBack;}
         
         if (Input.GetKey(KeyCode.R)) {yThrust = thrustUp;}
         else if (Input.GetKey(KeyCode.F)) {yThrust = -thrustDown;}
         
         if (Input.GetKey(KeyCode.D)) {xThrust = thrustRight;}
         else if (Input.GetKey(KeyCode.A)) {xThrust = -thrustLeft;}
         
         if (Input.GetKey(KeyCode.UpArrow)) {xRot = thrustPitch;}
         else if (Input.GetKey(KeyCode.DownArrow)) {xRot = -thrustPitch;}
         
         if (Input.GetKey(KeyCode.RightArrow)) {yRot = thrustYaw;}
         else if (Input.GetKey(KeyCode.LeftArrow)) {yRot = -thrustYaw;}
         
         if (Input.GetKey(KeyCode.Q)) {zRot = thrustRoll;}
         else if (Input.GetKey(KeyCode.E)) {zRot = -thrustRoll;}
         
         //get the amount of mouse steering
         mouseXSteer = Input.GetAxis("Mouse X");
         mouseYSteer = Input.GetAxis("Mouse Y");
 
         //acceleration
         rb.AddRelativeForce(xThrust, yThrust, zThrust);
         
         //rotation
         Vector3 rotation = new Vector3(xRot, yRot, zRot);
         rb.AddRelativeTorque(rotation);
         
         Vector3 mouseRotation = new Vector3(-mouseYSteer, mouseXSteer, 0);
         rb.AddRelativeTorque(mouseRotation * mouseSteerSensitivity);

Now I want to apply force in the negative moving direction to gradually slow down the ship when the inertia dampeners are active. It works when I am facing forwards in word space, but as soon as I turn the ship around, the ship starts moving in the wrong directions. This is weird as I use transform.InverseTransformDirection(rigidbody.velocity) to get the local moving direction.

The code for the inertia dampening:

         //check for inertiaDampeners toggle
         if (Input.GetKeyDown(KeyCode.LeftControl)) {
             inertiaDampeners = !inertiaDampeners;
         }
 
         //add inertia damener force
         if (inertiaDampeners == true) {
             Vector3 currentVel = transform.InverseTransformDirection(rb.velocity);
 
             if (currentVel.x > 0) {
                 rb.AddRelativeForce(transform.right * -thrustLeft);
             } else if (currentVel.x < 0) {
                 rb.AddRelativeForce(transform.right * thrustRight);
             }
             
             if (currentVel.y > 0) {
                 rb.AddRelativeForce(transform.up * -thrustDown);
             } else if (currentVel.y < 0) {
                 rb.AddRelativeForce(transform.up * thrustUp);
             }
             
             if (currentVel.z > 0) {
                 rb.AddRelativeForce(transform.forward * -thrustBack);
             } else if (currentVel.z < 0) {
                 rb.AddRelativeForce(transform.forward * thrustForward);
             }
         }

Does anyone know how to fix this? Or did I misunderstand transform.InverseTransformDirection(rigidbody.velocity).

Many thanks in advance!