2D Mouse Drag and Rotate Object Problem

Hi guys,

I am currently making a mobile, physics, rube goldberg type game for class. I have the ability to tap an object where a circle pops up around the object notifying the player he or she can move and rotate that object. When you click on the rotate icon, the object follows the mouse great, but when I let go the rotation messes up.

I wanted it to where the the UI follows the rotation with the object, but once let go it returns to its starting position. However, that seems to be messing with my objects rotation and can't quite figure out how. I know it is because my mouse cursor and base angle has changed, I just can't quite figure out how to figure this bug out.

Here is a video of what I am currently experiencing here

Here is my code for this mechanic. I am using two classes, one called Interactable for objects I can interact with, and a Controller class.

Controller class code:

             // If the player grabbed the rotate button
             case "RotateCircle":
                 // If an object is currently selected
                 if (currentSelection != null)
                 {
                     // If Mouse Button is being held on Rotate Circle icon
                     if (Input.GetMouseButton(0))
                     {
                         // Set current selected object to rotating
                         currentSelection.IsRotating = true;

                         // Set alpha of UI to 0.4f if rotating
                         InteractableUI.GetComponent<SpriteRenderer>().color = new Color(1f, 1f, 1f, 0.4f);
                         PressedCircle.GetComponent<SpriteRenderer>().color = new Color(1f, 1f, 1f, 0.4f);
                         ExitCircle.GetComponent<SpriteRenderer>().color = new Color(1f, 1f, 1f, 0.4f);

                         (!!!!*** Right here is where I find the Base Angle for rotation ***!!!!)
                         // Find Base Angle of Rotation for Object
                         if (currentSelection.BaseAngle == 0)
                         {
                             Vector3 pos = Camera.main.WorldToScreenPoint(currentSelection.transform.position);
                             pos = Input.mousePosition - pos;
                             currentSelection.BaseAngle = Mathf.Atan2(pos.y, pos.x) * Mathf.Rad2Deg;
                             currentSelection.BaseAngle -= Mathf.Atan2(transform.right.y, transform.right.x) * Mathf.Rad2Deg;
                         }
                     }
                 }
                 break;

When the object is clicked on, the Object clicked on becomes the parent for the InteractableUI object:

                      // Change position of Interactable UI to Current Selection and set parent
                     InteractableUI.transform.position = hit.transform.position;
                     InteractableUI.transform.SetParent(hit.transform);
                     InteractableUI.SetActive(true);

Interactable class code:

 void OnRotateDrag()
 {
     if (IsRotating)
     {
         if (Input.GetMouseButton(0))
         {
             Vector3 pos = Camera.main.WorldToScreenPoint(transform.position);
             pos = Input.mousePosition - pos;
             float ang = Mathf.Atan2(pos.y, pos.x) * Mathf.Rad2Deg - baseAngle;
             transform.rotation = Quaternion.AngleAxis(ang, Vector3.forward);
         }
         else {
             IsRotating = false;
         }
     }
 }

Then, in update of the controller class I check if Mouse button has been released and reset the rotation.

     if (Input.GetMouseButtonUp(0))
     {
         InteractableUI.transform.rotation = Quaternion.identity;
     }

Thanks in advance for the help! :)