Raycast direction change problem

I'm trying to fix this for like three hours.

I have a moving Cube object(which doesn't rotate) and a few other stationary objects. When Cube runs into another object its supposed to bounce off and go another way. I use a ray to detect a collision with another object and then change Cube's movement direction via Vector3.Reflect.

Everything basically works once.

Movement and debug ray change their directions on a first collision, but on a second one nothing happens. I tried the same without changing the movement direction and the Cube went through a few other objects saying "Hit!" in console a few times for every thing(for every frame before it goes through the object as i understand). So basically Raycast does work, but doesn't update to new direction vector...

The code has been changed quite a few times during the last 3 hours and now some things might look redundant, but anyway this is what i got:

     bool GameStarted = false;
     Vector3 Velocity;
     float speed;
     Vector3 rayOrigin;
     Vector3 rayDirection;
     float rayLength;
 
     void Start () {
         Velocity = transform.right;
         rayDirection = Velocity;
         speed = 15;
     }
 
     void FixedUpdate () {
         if (GameStarted == true) {
             transform.Translate(Velocity*speed*Time.deltaTime);
         }
     }
 
     void Update () {
         if (GameStarted == true) {
 
             rayOrigin = transform.position;
             rayDirection = Velocity;
             rayLength = (Velocity.x + Velocity.z);
             
             RaycastHit hitInfo = new RaycastHit(); 
             Ray ray = new Ray(rayOrigin, rayDirection);
 
             Debug.DrawRay(rayOrigin, Velocity, Color.red, 0, false);
             
             if (Physics.Raycast(ray, out hitInfo, rayLength)) {
                 Velocity = Vector3.Reflect(Velocity, hitInfo.normal);
                 rayDirection = Vector3.Reflect(rayDirection, hitInfo.normal);
                 print("Hit!");
             }
         }
         
 
         if (Input.GetKeyDown(KeyCode.Space)) {
             GameStarted = true;
             print("Vel: " +Velocity +" Dir: " +rayDirection +" Length: " +rayLength);
         }
 
 
 
     }

What im doing wrong here and how to make it work?