Rotate main camera using phone gyroscope with +Z-axis always aligned to North direction and +X-axis aligned to East direction

Hello fellow Unity Developers! I am facing a small issue with rotating main camera using system gyroscope. I am trying to fix the unity's XYZ such that +Z-axis is always aligned to North direction and +X-axis aligned to East direction. By fixing the unity's XYZ I mean that whenever I start the app by holding the phone in any direction or orientation, the reference 3D objects on +Z-axis and +X-axis should always be seen in the direction of true north and true east respectively.

I have come up with below code with a reference from this script . Now my code gives desired output when I start the app holding it up-right in landscape mode in *almost all orientations(*by almost all I mean, of all the orientations possible by placing the phone in a VR device). Only in case of 2 orientations viz, placing it flat on a table with the screen facing up and the screen facing the table, it behaves a bit randomly. This happens in an android phone(Xiomi Redmi Note 3) and an iOS phone(iPhone 5S) both, the special cases show reference objects in random directions when app started with phones placed flat with screen facing up or down.

I am using below code to rotate the camera transform:

 using UnityEngine;
 using System.Collections;
 
 public class GyroCameraManager : MonoBehaviour
 {
     // STATE
     private float _initialYAngle = 0f;
     private float _appliedGyroYAngle = 0f;
     private float _calibrationYAngle = 0f;
     private Transform _rawGyroRotation;
     private float _tempSmoothing;
 
     // SETTINGS
     [SerializeField] private float _smoothing = 0.1f;
 
     private IEnumerator Start()
     {
         Input.gyro.enabled = true;
         Input.compass.enabled = true;
         Application.targetFrameRate = 60;
 
         _rawGyroRotation = new GameObject("GyroRaw").transform;
         _rawGyroRotation.parent = transform;
         _rawGyroRotation.position = transform.position;
         _rawGyroRotation.rotation = transform.rotation;
 
         // Wait until gyro is active, then calibrate to reset starting rotation.
         yield return new WaitForSeconds(1);
 
         _initialYAngle = Input.compass.trueHeading;
         StartCoroutine(CalibrateYAngle());
     }
 
     private void Update()
     {
         ApplyGyroRotation();
         ApplyCalibration();
 
         transform.rotation = Quaternion.Slerp(transform.rotation, _rawGyroRotation.rotation, _smoothing);
     }
 
     private IEnumerator CalibrateYAngle()
     {
         _tempSmoothing = _smoothing;
         _smoothing = 1;
         _calibrationYAngle = _appliedGyroYAngle - _initialYAngle; // Offsets the y angle in case it wasn't 0 at edit time.
         yield return null;
         _smoothing = _tempSmoothing;
     }
 
     private void ApplyGyroRotation()
     {
         _rawGyroRotation.rotation = Input.gyro.attitude;
         _rawGyroRotation.Rotate(0f, 0f, 180f, Space.Self); // Swap "handedness" of quaternion from gyro.
         _rawGyroRotation.Rotate(90f, 180f, 0f, Space.World); // Rotate to make sense as a camera pointing out the back of your device.
         _appliedGyroYAngle = _rawGyroRotation.eulerAngles.y; // Save the angle around y axis for use in calibration.
     }
 
     private void ApplyCalibration()
     {
         _rawGyroRotation.Rotate(0f, -_calibrationYAngle, 0f, Space.World); // Rotates y angle back however much it deviated when calibrationYAngle was saved.
     }
 }

I am using unity 5.6.3p4. I do not have any other iOS device but I have tried in couple of android phones and the results are same. Please tell me what could be wrong with the code? Please help!