Android Swipe In 3D World

I am working on an AR application using Unity3D and the Vuforia SDK for Android. The way the application works is a when the designated image(a frame marker in our case) is recognized by the camera, a 3D island is rendered at that spot. Currently I am able to detect when/which objects are touched on the model by raycasting. I also am able to successfully detect a swipe using this code:

 if (Input.touchCount > 0) 
 {
 
             Touch touch = Input.touches[0];
 
             switch (touch.phase) 
             {
                 case TouchPhase.Began:
                 
                     couldBeSwipe = true;
 
                     startPos = touch.position;
 
                     startTime = Time.time;
 
                 break;
 
             
                 case TouchPhase.Moved:
 
                     if (Mathf.Abs(touch.position.y - startPos.y) > comfortZoneY) 
                     {
 
                         couldBeSwipe = false;
 
                     }
                     //track points here for raycast if it is swipe
                 break;
             
 
                 case TouchPhase.Stationary:
 
                     couldBeSwipe = false;
 
                 break;
 
             
 
                 case TouchPhase.Ended:
 
                     float swipeTime = Time.time - startTime;
 
                     float swipeDist = (touch.position - startPos).magnitude;
                     swipeCounter.guiText.text = "Touch phase ended. cbs: --" + couldBeSwipe;
                 
                     if (couldBeSwipe && (swipeTime < maxSwipeTime) && (swipeDist > minSwipeDist)) 
                     {
 
                         // It's a swiiiiiiiiiiiipe!
 
                         float swipeDirection = Mathf.Sign(touch.position.y - startPos.y);
     
                         // Do something here in reaction to the swipe.
                         swipeCounter.IncrementCounter();
                     }
                 
                 break;
                 
                 
             }
             
             touchInfo.SetTouchInfo
                 (Time.time-startTime,(touch.position-startPos).magnitude,Mathf.Abs (touch.position.y-startPos.y));
         }

Thanks to andeeeee for the logic. But I want to have some interaction in the 3D world based on the swipe on the screen. I.E. If the user swipes over unoccluded enemies, they die. My first thought was to track all the points in the moved TouchPhase, and then if it is a swipe raycast into all those points and kill any enemy that is hit. Is there a better way to do this? What is the best approach?

Thanks for the help!