Ideas on making a Lerp(Camera1, Camera2) ?

I am looking for some ideas on making a lerp function between 2 cameras on different parts of the scene graph. This feature is in regards to a shouldered gun. Meaning, I child a camera to a gun, and when I shoulder the weapon, the gun is moved such that it has the same orientation as seen from the gun camera's perspective.

I almost have this working: I have translation working but I am missing a key math step when with rotations.

So to give some more concrete details, my scene graph is as follows:

RootScene    
   -> Player
     -> MainCamera
       -> Weapon
         -> ShoulderedCam

Recap: I want 'weapon/gunobject' to be moved in a way such that when you are looking through MainCamera, it looks the same as if looking through ShoudleredCam.

//And I have my script as follows: GameObject GunObject = ...; Camera playerCam = ...; Camera gunCam = (Camera)GunObject.transform.FindChild("ShoulderedCam").GetComponent(typeof(Camera));

 // Troubles ?        

StartShoulderRot = playerCam.transform.localRotation; EndShoulderRot = StartShoulderRot*Quaternion.Inverse(gunCam.transform.localRotation);

And in the tick I just lerp betwen start -> end, and slerp between rotStar and rotEnd...

I would appreciate any pointers on which transform I am missing here :)