Make 2 object not go into each other when modifying one object's position

Hi I'm not sure how the rooms work, so I put it on default..

I'm basically trying to get my "character" (for now it's an egg) to move all around the inside of a pipe; https://imgur.com/a/eQAXHsA

The code I tried to make was the following:

 public class EggControl : MonoBehaviour
 {
     public Transform midPipe;
     public float rayDownDistance;
     public float rayUpDistance;
 
     // Update is called once per frame
     void Update()
     {
         RaycastHit hitDown;
         RaycastHit hitUp;
 
         //Raycast from object perspective-------------------------------------------
 
         if (Physics.Raycast(transform.position, transform.TransformDirection(Vector3.down), out hitDown) &&
             Physics.Raycast(transform.position, transform.TransformDirection(Vector3.up), out hitUp))
         {
             rayDownDistance = hitDown.distance;
             rayUpDistance = hitUp.distance;
 
 
             Debug.DrawRay(transform.position, hitDown.point, Color.red);
             Debug.DrawRay(transform.position, hitUp.point, Color.green);
 
             transform.localPosition = new Vector3(hitDown.point.x, transform.position.y, transform.position.z);
         }
 
         if (Input.GetKey(KeyCode.A))
         {
             transform.Rotate(0, 0, -1);
         }
         if (Input.GetKey(KeyCode.D))
         {
             transform.Rotate(0, 0, 1);
         }
 
         Debug.Log("update is going through");
   }
 }

So basically I change the x.position of the main object when the ray is cast from underneath, but once it gets onto another wall it has to change both x and y, but if I put the egg precisely on the hit.point it'll get stuck in the pipe instead. Is there a way so that the main object stays outside the pipe and never actually goes into it? Or is there a better way to approach this?

I have tried to get the middle of the pipe and cast a ray from there, but than first of all, the ray gets cast into the wrong direction somehow; https://imgur.com/a/eQAXHsA . And second the movement is completely incorrect and I can't tell why (it starts at the bottom => goes up automatically => resets down => goes up automatically => resets down => goes up automatically => resets down => etc.

 //Raycast from midPipe perspective---------------------------------------------------------------
 
         if (Physics.Raycast(midPipe.position, midPipe.TransformDirection(midPipe.up * -1), out hitDown))
         {
             rayDownDistance = hitDown.distance;
 
             Debug.DrawRay(midPipe.position, hitDown.point, Color.red);
 
             transform.position = new Vector3(hitDown.point.x, hitDown.point.y, transform.position.z);
         }
 
         if (Input.GetKey(KeyCode.A))
         {
             midPipe.Rotate(0, 0, -1);
         }
         if (Input.GetKey(KeyCode.D))
         {
             midPipe.Rotate(0, 0, 1);
         }

Anybody got a clue on how I could approach this?