Enable NavMeshObstacle when all units are out of the way

Hello, I am working through creating an RTS and I've hit a wall with yet another problem while building a building.

I need to be able to place a building foundation on the ground for my units to begin working on. I can detect when I place the building over one of my units. I check to see if the Object has the Unit class, and check for a boolean "isPlayerObject". This works fine, and I can even tell the unit to move to the edge of the building's bounds.

The problem is with my NavMeshObstacle. I don't want the act of placing the building to push my Unit agent out of the bounds, I want them to walk to the edge. So I need some mechanism to detect when all units have moved out of the way.

Here's what I've got so far, but I feel like I'm severely overcomplicating this, AND it really only means the first unit to slow to a velocity of .1 triggers the NavMeshObstacle to be enabled... meaning other units may get pushed out of the bounds at the last second. What I really need to know is how to wait to enable the NavMeshObstacle until all units in the unitsToMove array are TRUE.

      unitsToMove = Physics.OverlapBox(buildLoc, buildExtent); // Create an array of colliders that are currently in the building
     for(int i = 0; i < unitsToMove.Length; i++)
     {
         if (unitsToMove[i].GetComponent<Unit>() && unitsToMove[i].GetComponent<ObjectInfo>().isPlayerObject) //Are they a unit, and do they belong to the player?
         {
             Unit unitToMove = unitsToMove[i].GetComponent<Unit>();

             destination = new Vector3(Random.Range(buildMax.x, buildMin.x), 0, buildMax.z); //Get random range between buildMin & buildMax for X, Y, Z

             unitToMove.Move(unitToMove.agent, destination); //Call the Unit's Move function and move to the random spot along the edge of the building.
             StartCoroutine(MovingOutOfBuilding(buildingCollider)); //Constantly check to see if this unit has slowed down.
         }
     }

  IEnumerator MovingOutOfBuilding(NavMeshAgent agent, Collider buildingCollider)
 {
     while (true)
     {
         yield return new WaitForSeconds(1);
         if (agent.velocity.magnitude < .1f)
         {
             buildingCollider.GetComponent<NavMeshObstacle>().enabled = true;
             yield break;
         }
     }
 }

If you can't tell I'm pretty new at this, and really hopeful someone with some experience can point me in the right direction. Thanks in advance!!