How to fix Position relative to Raycast lenghts

Hi, my goal is to get the current Position of an Object in an real Room and visualize that in Unity. I've reconstucted the Room(simple 3x5x2m Room) via Unity. The Object has an tilt sensor and six ultasonic sensors. My Idea was to rotate the Objects similar to the tilt sensor and then get Raycasts in all six directions and compare those to what the actual distance in that direction should be (ultrasonic sensors). Then i'll add those differences to one new Vector 3 and add this to my transform.position. Unfortunately my Object keeps jumping around and is not just fixed to one position.

My Code looks like this

 void Update () {
     //ROTATE DRONE
     transform.rotation = Quaternion.Euler(pitch, compassHeading - configLoaderBehaviourScript.roomRotation, roll);
     //UPDATE DIRECTIONS
     directions[0] = transform.forward;
     directions[1] = -transform.right;
     directions[2] = transform.right;
     directions[3] = -transform.up;
     directions[4] = -transform.forward;
     directions[5] = transform.up;

     List<Color> colors = new List<Color>() {Color.red,Color.blue,Color.green,Color.yellow, Color.white, Color.magenta };
     //ULTRASONIC
     deltaPosition = new Vector3();

     Debug.DrawRay(transform.position, (Vector3)directions[0] * (float)distances[0], colors[0], 2, false);
     Debug.DrawRay(transform.position, (Vector3)directions[1] * (float)distances[1], colors[1], 2, false);
     Debug.DrawRay(transform.position, (Vector3)directions[2] * (float)distances[2], colors[2], 2, false);
     Debug.DrawRay(transform.position, (Vector3)directions[3] * (float)distances[3], colors[3], 2, false);
     Debug.DrawRay(transform.position, (Vector3)directions[4] * (float)distances[4], colors[2], 2, false);
     Debug.DrawRay(transform.position, (Vector3)directions[5] * (float)distances[5], colors[3], 2, false);



     for (int i = 0; i < 6; i++)
     {

             if (((float)distances[i]) >= 0)
             {
                 RaycastHit hit;
                 Ray ray = new Ray(transform.position, (Vector3)directions[i]);
                 if (configLoaderBehaviourScript.debugMode)
                 {
                     Debug.DrawRay(transform.position, (Vector3)directions[i] * (float)distances[i], colors[i], 2, false);
                     createLine(transform.position, (Vector3)directions[i] * (float)distances[i]);
                 }
                 if (Physics.Raycast(ray, out hit))
                 {
                 //30 == 3 Meters => MAxRange of Sensors
                     if ((float)distances[i] < 30)
                     {
                         deltaPosition += (Vector3)directions[i] * (hit.distance - (float)distances[i]);
                     }
                     else if (hit.distance < 30)
                     {
                         deltaPosition += (Vector3)directions[i] * (hit.distance - 30);
                     }

                 }
                 else
                 {
                     //NOHIT -> Not in Room anymmore -> reset position
                     transform.position = new Vector3(0f, 10f, 10f);
                     break;
                 }
         }

     }

     transform.position += deltaPosition;
     [...]
     }

First 2 Frames (The Colours for back and up are wrong here) After that it just seems to move constantly.

Does anyone have an Idea how to fix this?

Best Regards