Problem raycasting to quads at certain angles

Hello,

I'm trying to teleport my player to a face of a cube made out of quads. And I want to align the player's transform.up with the face of the quad. The thing is, this works well for some faces and doesn't for others. My script works like this:

1. I set the player's position to be the same as the quad.

2. I ray cast in all 6 directions (vector3.up, vector3.down, ect...)

3. If the ray cast hits, that means the quad is in that direction and I change the transform.up of the player so that he is standing/aligned on/with the quad.

This works for some faces but not for others. Some quads are not hit by my raycasting. However, if I put one of the quads that is not being hit, at a slight angle (say, Rotation(0,183,0) instead of (0,180,0)) then the raycast hits that quad and all works perfectly. And I don't know why.

I've tried everything, even offsetting the raycast angle, but it still doesn't work.

Here's the code:

 Vector3[] vectorOrientations = new Vector3[6];
 
  void Awake ()
     {
         //Populating the array with all the Vector3.orientations.
         vectorOrientations[0] = Vector3.up;
         vectorOrientations[1] = Vector3.down;
         vectorOrientations[2] = Vector3.right;
         vectorOrientations[3] = Vector3.left;
         vectorOrientations[4] = Vector3.forward;
         vectorOrientations[5] = Vector3.back;
 
         playerHeight = transform.localScale.y;
     }
 public void MoveDestination(Transform destinationFace)
     {
         //send the player to another face
         transform.position = destinationFace.transform.position;
 
         RaycastHit hit;
 
         //Cycle through every orientation..
         for (int i = 0; i < vectorOrientations.Length; i++)
         {
             //..and do a raycast to see if it hits anything..
             if (Physics.Raycast(transform.position, vectorOrientations[i], out hit, 3.0f, 1, QueryTriggerInteraction.Ignore))
             {
                 //..other than the player.
                 if (hit.transform.tag != "Player")
                 {
                     //align the vector up of the player with the quad
                     transform.up = hit.transform.forward * -1;
 
                     //raise the player from the floor by adding their height to themselves
                     transform.Translate(Vector3.up * playerHeight);
                     
                 }
             }
         }
     }

(The small cubes are paired triggers that send the player back and forth to the other faces of the cube)