How to accept swipe input when already touching with one finger.

I am using the "new" input system and have 2 joysticks on screen with the on-screen stick component script. They work great even when handling both joysticks being used at once.

My layout for this mobile game is the top half of the screen is for swiping, the bottom left is a canvas for the movement joystick, and the bottom right is a canvas for the aiming joystick.

My swipe detection also works well on its own. However, I didnt recognize this until building to a phone today but my swipe detection script is currently overridden by touching the joysticks and it kills the script since I'm already touching the screen. How can i change my script or better utilize the input system to handle the swipe input?:

 private void OnEnable()
     {
         inputManager.OnStartTouch += SwipeStart;
         inputManager.OnEndTouch += SwipeEnd;
     }
 
     private void OnDisable()
     {
         inputManager.OnStartTouch -= SwipeStart;
         inputManager.OnEndTouch -= SwipeEnd;
     }
 
     private void SwipeStart(Vector2 position, float time)
     {
         if (position.y < Screen.height / 2f)
         {
             return;
         }
         startPosition = position;
         startTime = time;
     }
 
     private void SwipeEnd(Vector2 position, float time)
     {
         endPosition = position;
         endTime = time;
         DetectSwipe();
     }
 
     private void DetectSwipe()
     {
         if (Vector3.Distance(startPosition,endPosition) >= minDistance && (endTime - startTime) <= maxTime)
         {
             playerController.SwapWeapons();
             Debug.DrawLine(startPosition, endPosition, Color.red,5f);
         }
     }
 }