Problem fixing quaternions

To begin with, calibrate your system by noting the first quaternion that gets returned, and storing its inverse. Then for subsequent quaternions, multiply them by that inverse before assigning them to the camera. This means that the camera's initial position should not jump when you first start applying the quaternions, so if you ensure the head tracker's initial orientation is sensible when you calibrate then it makes the next steps easier.

Next, carefully keeping the head tracker in that initial position, rotate it slightly to the left and observe what happens to the camera - it should rotate about one of the axes, but possibly not the correct one, so it could move left, right, up, down, or roll clockwise or anticlockwise. Then return the head tracker to its initial position again, and rotate it slightly upwards, and again note which way the camera moved. Finally, after returning the head tracker to its initial state, roll it about its forwards axis, and again note what happens to the camera.

If you post those results here - basically which axis the camera rotated around, and in which direction, for each movement of the head tracker - then I might be able to suggest how you should convert the orientations between bases.

Okay. I did what you said. The results are:

Rotate device to the left over the table (person looking left) -> looks to the left

Rotate device to the righ over the table (person looking right)-> looks to the right

Rotate device upwards over the table (person looking up) -> skyline rotates anti-clockwise and becomes vertical

Rotate device downwards over the table (person looking down) -> skyline rotates clockwise and becomes vertical

Rotate device anti-clockwise (right side of the device lifts from the table) -> looks up

Rotate device clockwise (left side of the device lifts from the table)-> looks down

So, Rotate device upwards should do what Rotate device anti-clockwise is doing and Rotate device downwards should do what Rotate device clockwise is doing.

The code used now is:

  using UnityEngine;
     using System;
     using System.Collections;
     using System.IO.Ports;
     using System.$$anonymous$$anagement;
     
         
     public class YEIHeadTracking : $$anonymous$$onoBehaviour{
      
         public static SerialPort sp = new SerialPort("CO$$anonymous$$4");
      
         public static byte[] send_bytes = {0xf7, 0x00, 0x00};
         public static byte[] button_bytes = {0xf7, 0xfa, 0xfa};
         public static byte[] tare_bytes = {0xf7, 0x60, 0x60};
      
         public int counter = 0;
     
         public Quaternion InitialQuaternionInversed;
      
         
         void Start(){
     
                sp.BaudRate = 115200;
                sp.Parity = Parity.None;
                sp.DataBits = 8;
                sp.StopBits = StopBits.One;
      
            // Set the read/write timeouts
               sp.WriteTimeout = 100;
            sp.ReadTimeout = 100;
      
               sp.Open();
     
                sp.Write(send_bytes,0,3);
         }


         float bytesToFloat(byte[] raw_bytes, int offset){
                
         byte[] big_bytes = new byte[4];
                
         big_bytes[0] = raw_bytes[offset+3];
                big_bytes[1] = raw_bytes[offset+2];
                big_bytes[2] = raw_bytes[offset+1];
                big_bytes[3] = raw_bytes[offset+0];
                
         return BitConverter.ToSingle(big_bytes,0);
         }
     
      

         void Update(){
     
         counter++;
             
             byte[] read_bytes = new byte[16];
      
             int read_counter = 100;
             int byte_idx = 0;
             while (read_counter > 0){
                 try{
                     byte_idx += sp.Read(read_bytes, byte_idx, 1);
                 }
                 catch{
                     // Failed to read from serial port
                 }
                 if (byte_idx == 16){
                     break;
                 }
                 if (read_counter <= 0){
                     throw new System.Exception("Failed to read quaternion from port too many times." +
                         " This could mean the port is not open or the $$anonymous$$ono serial read is not responding.");
                 }
                 --read_counter;
             }
      
            // Convert bytes to floats
            float x = bytesToFloat(read_bytes,0);
            float y = bytesToFloat(read_bytes,4);
            float z = bytesToFloat(read_bytes,8);
            float w = bytesToFloat(read_bytes,12);
     
     
         if (counter == 1) {
     
             Quaternion InitialQuaternion = new Quaternion (x, y, z, w);
             InitialQuaternionInversed = Quaternion.Inverse (InitialQuaternion);
     
         }
     
     
         Quaternion SubsequentQuaternion = new Quaternion(x,y,z,w);
         SubsequentQuaternion = SubsequentQuaternion * InitialQuaternionInversed;
     
         Camera.main.transform.rotation = SubsequentQuaternion;
      
      
            // Send command
            sp.Write(send_bytes,0,3);
         }
     }

That's good - so your camera is behaving as if it was looking along the device's right vector. To correct that you just need to multiply additionally by a 90-degree rotation about the Y axis, i.e. Quaternion.Euler(0, 90, 0) or possibly -90. Try each and see if either works.