Drag object by Touch quickly

Hi all, I checked a lot of question about how drag an object by touch, but I'm not been able to solve my problem. I've a 2D project on Android and the problem is that when I move my finger very quickly I lose contact with the object dragged and it stops. Here are two simple examples of how I tried, one using raycast and one with collider. Still don't know which is the best method and how can I solve to keep the object "up to date" with the finger?

Edit : This is my new code that works almost perfectly, but still laggy. I'll glad if anyone knows how to improve this code.

     public Ray ray;
     RaycastHit hit;
     public Vector2 pos;
     public Transform toDrag ;
     public bool dragging = false;
     public Vector2 offset;
     public Vector2 v2;
 
     void FixedUpdate() {
 
         foreach (Touch touch in Input.touches) {
             Ray ray = Camera.main.ScreenPointToRay(touch.position);
             pos = Camera.main.ScreenToWorldPoint(new Vector2(Input.GetTouch(0).position.x, Input.GetTouch(0).position.y));
             
             RaycastHit2D hit = Physics2D.Raycast(ray.origin, ray.direction);
 
             if(touch.phase == TouchPhase.Began)
             {
                 if(hit.collider != null)
                 {
                     toDrag = hit.transform;
                     v2 = new Vector2(pos.x, pos.y);
                     offset = new Vector2 (toDrag.position.x, toDrag.position.y ) - v2;
                     dragging = true;
                 }

             }
             if(touch.phase == TouchPhase.Moved)
             {
                 
                 if (dragging)
                 {
                     v2 = new Vector2(pos.x, pos.y);
                     toDrag.position = v2 + offset;
                 }
                 // optional, if the touch is moving and  it is encountered with an object
                 else if (hit.collider != null && dragging == false)
                 {
                     toDrag = hit.transform;
                     v2 = new Vector2(pos.x, pos.y);
                     offset = new Vector2 (toDrag.position.x, toDrag.position.y ) - v2;
                     toDrag.position = v2 + offset;
                     dragging = true;
                 }
 
             }
 
 
             if(touch.phase == TouchPhase.Ended)
             {
                 dragging = false;
             }
         }
     }