Rigidbody goes wonky after collision

Hello, I'm having a strange issue where my rigid bodies get all out of whack after a collision with something.

This is what it looks like normally (Before the collision with that capsule to the right) As you can see, the blue and yellow lines are pointing forward and everything looks good.

And this is what it looks like after the collision. Everything is askew, and the rigidbody applies force in a strange direction seen here (The yellow line is the direction it applies forward force). Its especially strange since that box on the bottom should have the exact same rotation as the box above it. I've tried using different types of colliders but they all have the same issue.

The thing is, it kinda makes sense why it is doing this. It's like trying to push a box past an object, but the corner of the box gets caught on the edge of the other object, causing the box to rotate. What I don't understand is why it doesn't reset itself properly.

Here is the code for the rigidbody:

 void FixedUpdate()
     {
         //Turning, pitching, and rolling
         targetRotation.x = Mathf.Clamp(targetRotation.x, -maximumPitch, maximumPitch);
         targetRotation.z = Mathf.Clamp(targetRotation.z, -maximumRoll, maximumRoll);
 
         currentRotation.x = Mathf.SmoothDamp(currentRotation.x, targetRotation.x, ref rotationV.x, lookSmoothDamp);
         currentRotation.y = Mathf.SmoothDamp(currentRotation.y, targetRotation.y, ref rotationV.y, lookSmoothDamp);
         currentRotation.z = Mathf.SmoothDamp(currentRotation.z, targetRotation.z, ref rotationV.z, rollSmoothDamp);
 
         transform.rotation = Quaternion.Euler(currentRotation);
 
         rigidbody.AddRelativeForce(0, 0, forward * currentSpeed);
 
     }

And some input code from the player:

 void Update()
 {
         targetRotation.y += Mathf.Clamp(Input.GetAxis("Mouse X"), -maxTurnSpeed, maxTurnSpeed) * lookSensitivity;
         targetRotation.x += Mathf.Clamp(Input.GetAxis("Mouse Y"), -maxPitchSpeed, maxPitchSpeed) * -lookSensitivity;
         targetRotation.z = -rollSpeed * Input.GetAxis("Mouse X") * forward;
 
         forward = Input.GetAxis("Forward");
 }