How to flick a game object using touch and raycasters ?

I've been searching for a solution for my problem for a while on the web, but i didn't find anything that work properly

What i want to do The game is in 2D mode. I have a bouncing ball and I want to let the user flick it with his finger (using a swipe gesture or whatever). The ball shouldn't start moving unless the player flick touch the ball. I don't want the ball to move if the player flicks anywhere on the screen, only if the flick touch the ball

What i've done so far :

I've tried to write a script which would detect the touch starting position and the touch end position, and then cast a ray from that point and check if it hit the ball to make sure that the flick touch the ball. Then the script calculate the direction vector by substrating the start position from the end position, and then give the ball a velocity vector which is the direction vector

The code inside my update methode :

 if (Input.touchCount > 0) {
                         Touch t = Input.touches [0];
                         if (t.phase == TouchPhase.Began) {
                                 start = Camera.main.ScreenToWorldPoint (new Vector3 (t.position.x, t.position.y, 0));
                         }
             
                         if (t.phase == TouchPhase.Ended || t.phase == TouchPhase.Canceled) {
                 end = Camera.main.ScreenToWorldPoint(new Vector3(t.position.x,t.position.y,0));
                 direction = end-start ;
                 Ray ray = Camera.main.ScreenPointToRay(new Vector3(end.x,end.y,0.00f));
                 RaycastHit hit;
                 if ( Physics.Raycast(ray, out hit,1000.00f))
                 {
                     print("touched"+hit.transform.gameObject.name);
                     rigidbody2D.velocity = direction;
                 }
             }
                 }

What's the problem ?

The problem is that the code doesn't work.

When i launch the game in the Unity Remote 4 app in my android device and try to flick the ball it doesn't move, the inspector shows that the the scripte is calculating the start & end positions and the direction vector, but the ball doesn't move, it stays in the same position even when the flick touch the ball. I am thinking that the raycast is not hitting the ball, but why ? How should i fix it ? is there a better way to do it ?

Since the ball have a 2D circle collider how to make it detect collision with the player swipe ?

Thank's !