Issue with Rotation behaviour of PID Controller when rotating to look at mouse position

So I've put together my own PID controller from a handful of different scripts I've found to rotate my player towards the mouse location with addTorque and it actually works, kinda.

The issue is that if the mouse is pointing above the player or to the right of the player it won't rotate to that point, but will only rotate towards a similar point on the other side. It's like the script is reading negative coordinates as positive and rotating the rigidbody towards the positive coordinate instead.

I'm not sure thats the whole issue though, as I'm using some error equations and plugging them into the addtorque line at the bottom of the script in a way that doesn't quite make sense to me, I feel like I should only be adding torque on the y-axis, and that torque should be a function of the x and z error values but my brains fried at this point. Just using the xerr or zerr on the y axis of the addTorque line doesn't work either, but I expected that.

Since the description of the problem is terrible here's a gif describing the behaviour: https://media.giphy.com/media/1vZapFYleh0mQmp4sc/giphy.gif

And here's my code:

 using UnityEngine;
 using System.Collections;
 
 [System.Serializable]
 public class PIDController
 {
 
     [Tooltip("Proportional constant (counters current error)")]
     public float Kp = 0.01f;
 
     [Tooltip("Integral constant (counters cumulated error)")]
     public float Ki = 20f;
 
     [Tooltip("Derivative constant (fights oscillation)")]
     public float Kd = 1f;
 
     [Tooltip("Current control value")]
     public float value = 10;
 
     private float lastError;
     private float integral;
 
     public float Update(float error)
     {
         return Update(error, Time.fixedDeltaTime);
     }
 
     public float Update(float error, float dt)
     {
         float derivative = (error - lastError) / dt;
         integral += error * dt;
         lastError = error;
 
         value = Kp * error + Ki * integral + Kd * derivative;
         return value;
     }
 }
 
 public class LookAtController : MonoBehaviour
 {
     Vector3 target;
     Vector3 mousePos;
     Rigidbody rb;
     Vector3 playerPos;
     Vector3 forward;
     float xerrA;
     float zerrA;
     
     public PIDController rotationPID;
 
     void Start()
     {
         rb = GetComponent<Rigidbody>();
     }
 
     public void FixedUpdate()
     {
 
         Ray ray = Camera.main.ScreenPointToRay(mousePos);
         RaycastHit hit;
         if (Physics.Raycast(ray, out hit, Mathf.Infinity))
         {
             target = hit.point;
             mousePos.x = Input.mousePosition.x;
             mousePos.y = Input.mousePosition.y;
             playerPos = rb.position;
             forward = rb.transform.forward;
 
             Vector3 cPR = forward;
             Vector3 dPR = (target - playerPos) / (target - playerPos).magnitude;
 
             float xerr = (dPR.x - cPR.x) * 100;
             float zerr = (dPR.z - cPR.z) * 100;
 
                 rb.AddRelativeTorque(rotationPID.Update(xerr), rotationPID.Update(zerr), 0);
         }   
     }
 }