How can I pragmatically move a navmesh rigid body when changing scenes?

I have an object with both rigid body and navmesh attached Which I want to set to a new location on scene change, but there is something weird going on with it. If I just try to set the position it will not have any effect, If I disable the navmesh (Stopping it does not work) it will correctly go to the new position, but it will then just fall through the floor, so I must also disable gravity on the rigidbody, but even this only works if there is some time between disabling it and the position change, which is really undesirable.

The bellow code sets the position, but only after long waits. How can I get it working without any waits involved?

 IEnumerator setPosition(Vector3 position, Quaternion rotation)
     {
         float time = 5.02f;
         this.GetComponent<Rigidbody>().useGravity = false;
 
         yield return new WaitForSeconds(time);
 
         this.GetComponent<NavMeshAgent>().enabled = false;
 
         yield return new WaitForSeconds(time);
 
         this.transform.position = position;
         this.transform.rotation = rotation;
 
         yield return new WaitForSeconds(time);
 
         this.GetComponent<NavMeshAgent>().enabled = true;
         this.GetComponent<NavMeshAgent>().Resume();
         this.GetComponent<Rigidbody>().useGravity = true;
     }