Rotating sprite through touch (storing current rotation)

I am trying to rotate a sprite based on a touch drag. The sprite is centered so I am getting the angle of the touch relative to the origin using arctan. Then, I set the sprite rotation to this angle using transform.rotation = Quaternion.AngleAxis(newAngle, Vector3.forward);

This works great but the issue is when I try to rotate the sprite again, the rotation jumps because I believe I have not accounted for the previous rotation of the sprite. I tried to get the rotation through transform.rotation.ToAngleAxis() but I can't figure out how to do this.

             if (touchPhase == TouchPhase.Began) {
                 Vector2 touchPos = new Vector2(originalPosition.x, originalPosition.y);
                 // Make sure you are touching this object
                 if (!collider2D == Physics2D.OverlapPoint(touchPos)) {
                     isTouched = true;
                 }
             } else if (!isTouched) {
                 return;
             } else if (touchPhase == TouchPhase.Moved) {
                 Vector3 dir = Camera.main.ScreenToWorldPoint(Input.GetTouch(0).position);
                 float newAngle = Mathf.Atan2(dir.y,dir.x) * Mathf.Rad2Deg;
                 transform.rotation = Quaternion.AngleAxis(newAngle, Vector3.forward);