quaternion - rotate camera based on original facing at scene start?

It's my first time really trying to do something with quaternion math. There are many tutorials out there showing how one can get a camera to rotate with the device's gyroscope, but I'm trying to do something one step farther.

After following various tutorials, the initial facing of the camera in my game is dependent on how I'm holding my phone at the time the scene loads. This is NOT what I want. I would like the game camera to always start as I have it set in the scene designer - for example if the player should start facing a fire, they should always start facing the fire no matter how they're holding the phone when the scene loads. Then as they turn the phone, the camera rotates away from the fire to show the rest of the scene. When they return to their original real world position, they're again looking at the fire.

Here is my latest attempt, where I try sampling the initial camera and gyroscope quaternions and then basing future rotations on the "delta" of the gyroscope's rotation. Put simply, it doesn't work. What am I doing wrong here? I assume it's a math problem.

 public class RotateByDeviceRotation : MonoBehaviour
 {
     private Quaternion originalObjectRotation;
     private Quaternion originalGyroRotation;
 
     // Start is called before the first frame update
     void Start()
     {
         if(SystemInfo.supportsGyroscope)
         {
             Input.gyro.enabled = true;
             this.originalObjectRotation = transform.rotation;
             this.originalGyroRotation = Input.gyro.attitude;
         }
     }
 
     // Update is called once per frame
     void Update()
     {
         if(Input.gyro.enabled)
         {
             transform.rotation = this.originalObjectRotation * GyroToUnity((this.originalGyroRotation * Quaternion.Inverse(Input.gyro.attitude)));
         }
     }
 
     private static Quaternion GyroToUnity(Quaternion q)  //from unity docs
     {
         return new Quaternion(q.x, q.y, -q.z, -q.w);
     }
 }