Physics.SphereCast with Vector3.Reflect returns weird results

Hello, I try to have a ray that simulates the position of a ball which will move and collide with others objects. I use the Physics.SphereCast and Vertex3.Reflect method to do this but I got bad results, I added a screenshot, in red it's the calculated ray and in green as it should be.

Here is my script to show the ray :

 // Create start ray
 Ray ray = new Ray(objectWorldPosition, direction);
 
 // Ray loop
 RaycastHit hitInfo;
 for (float length = 0.0f; length < 100.0f; length += hitInfo.distance) {
     // If raycast hits something
     if (Physics.SphereCast(ray, 0.5f, out hitInfo)) {
         // Calculate center position of ball
         Vector3 centerVector = hitInfo.point + (hitInfo.normal * 0.5f);
 
         // Draw line
         Debug.DrawLine(ray.origin, centerVector, Color.red);
 
         // Get vector from start to hit point
         Vector3 rayVector = centerVector - ray.origin;
 
         // Calculate reflective ray vector
         Vector3 reflectiveRayVector = Vector3.Reflect(rayVector, hitInfo.normal);
 
         // Update ray
         ray = new Ray(centerVector, reflectiveRayVector);
     } else {
         // Exit loop
         break;
     }
 }
 

The green ray is calculated with a Physics.RayCast and a Pythagore formula to find the center point of the ball from the hitpoint. It is more accuracy but with some angles that fails too and also on some situations where ball could not pass but ray does because it is thinner.

Can someone help me ? Thanks