I am having a weird issue with Quaternion.FromToRotation

So I am starting on a new project where I am building a random dungeon out of room prefabs. The first thing that happens is I instantiate a game object called "Start Room". This room has 4 exits as empty child game objects set in the locations that represent exits out of the room. Once the "Start Room"has been placed at 0,0,0 it spawns a new clone of a room prefab for each of its exits.

The new prefab that the "Start Room"has called has a single entrance and can have up to 3 exits as empty child game objects. The new prefab is then positioned at the "Start Room"exit and then Quaternion.FromToRotation is used to rotate the entrance of the newly spawned prefab to face the "Start Room"exit. Each Entrance and Exit child game object face to where their z axis is facing outward from the prefab so the entrance rotates on the y axis so that its z axis faces the z axis of the exit(if that makes sense)

This works fine for the most part but every so often one or two of the newly spawned prefabs are positioned correctly but the rotation is flipped 180 degrees on the z axis and I am not really sure why.

This is the script used to call the "Start Room" as well as position and rotate the newly spawned rooms:

  using UnityEngine;
 using System.Collections;
 using System.Collections.Generic;
 public class RoomPooler : MonoBehaviour
 {
     public GameObject startRoomPrefab;
     public GameObject roomPrefab;
     public static RoomPooler pooler;
 
     void Awake()
     {
         pooler = this;
     }
 
     void Start()
     {
         SpawnStartRoom();
     }
 
     void SpawnStartRoom()
     {
         GameObject startRoom = Instantiate(startRoomPrefab, new Vector3(0, 0, 0), Quaternion.identity) as GameObject;
         startRoom.GetComponent<StartRoomScript>().AssignExits();
     }
 
     public void SpawnAdditionalRooms(GameObject exit)
     {
         GameObject newRoom = Instantiate(roomPrefab);
         newRoom.GetComponent<Room>().AssignEntrance();
         AlignEntranceToExit(newRoom, newRoom.GetComponent<Room>().entrance, exit.transform.parent.gameObject, exit);
     }
 
     void AlignEntranceToExit(GameObject newRoom, GameObject entrance, GameObject exitParent, GameObject exitChild)
     {
         Vector3 v1 = newRoom.transform.position - entrance.transform.position;
         Vector3 v2 = exitChild.transform.position - exitParent.transform.position;
         newRoom.transform.position = exitChild.transform.position + v2.normalized * v1.magnitude;
         newRoom.transform.rotation = Quaternion.FromToRotation(v1, v2) * newRoom.transform.rotation;
     }
 
 

The "Start Room" uses this script to call a new room for each exit:

 using UnityEngine;
 using System.Collections;
 using System.Collections.Generic;
 
 public class StartRoomScript : MonoBehaviour
 {
     public List<GameObject> doors;
 
     public void AssignExits()
     {
         for (int i = 0; i < doors.Count; i++)
         {
             RoomPooler.pooler.SpawnAdditionalRooms(doors[i]);
         }
     }
 }

The Rooms use this script to assign a random entrance from its list of doors. For now it also attaches another script that just draws a line in OnDrawGizmos for each of the exits x,y and z. Also until I can figure out the issue the Room script does not call for an addition room.

 using UnityEngine;
 using System.Collections;
 using System.Collections.Generic;
 public class Room : MonoBehaviour
 {
     public List<GameObject> doors;
     public GameObject entrance;
 
 
     public void AssignEntrance()
     {
 
         int pickRandomDoorForEntrance = Random.Range(0, doors.Count);
         entrance = doors[pickRandomDoorForEntrance];
         entrance.name = "Entrance";
         doors.Remove(entrance);
         for (int i = 0; i < doors.Count; i++)
         {
             doors[i].gameObject.AddComponent<Exits>();
             doors[i].name = "Exit # " + i;
         }
 
     }
 
 
 }

I have attached a screen capture of what happens. The first is when everything aligns correctly but the second is where the rooms are misaligned because it has been rotated on the z axis as well as the y axis.

I could understand if it happens all the time but this does not. Sometimes it works correctly without an issue but sometimes it does not. These are also the only scripts in the project so there is no other scripting interfering.

Anyone have any thoughts?