Keeping 2D orthographic camera with zoom in level boundaries

I have a 2D orthographic camera that follows multiple targets and zooms in/out depending on how far or close the targets are to each other. The camera also always keeps itself centered between all targets.

This system works well, but often when players are towards the edge of the level boundaries, most of the camera goes off the edge of the level as well.

I am trying to have the camera free flow while its anywhere inside the level boundaries, but when it hits a boundry such as the right edge, then it will no longer move the camera's x position outside the bounds of the level.

I've spent many hours trying to find a solution to this problem. It is easy in my mind if the camera has NO zoom because you can just check the camera's X position against a boundries X and restrain it from moving past. However, once zoom is involved, I have no idea how to handle the calculations to keep the camera in boundaries...

I am using a 2D Toolkit camera and adjusting its zoomFactor property to adjust the camera zoom.

Any advice or help on the matter would be greatly appreciated!