Smooth, Frame independent, relative rotation?

Hello,
so my question seems relatively simple. I'm a noob with Unity (came from ue4) and basically what I'm trying to accomplish is to make the camera rotate smoothly, by 90deg when an input is triggered. For example: left trigger -> camera rotates 90deg to the left; right trigger -> camera rotates 90 to the right. Simple right? The catch, however, is that the camera has to rotate up and down as well, changing the world axis (because if the camera is rotated 90 on the x, then rotating it 90 on the y will not have the same effect as when x rotation is 0). I've tried many ways already but in the end it always messes up when rotating up or down.
This might be a little confusing but basically the camera must rotate completely relative to its prior rotation. What would be the best way to do that? Quaternions are very hard to understand and I couldn't really find any helpful posts explaining how to do this. Thank you!