Rendering the same Render Texture differently to separate cameras

I'm in the process of setting up an environment that imitates Portal 2 in Unity, with a few twists. One of them is that I need two 'players' in the same environment that is able to use and look through their own and the other player's portals. That means, in the same scene, I have two cameras that I can then switch between to control the different players.

I used the code from this repository for the base Portal functionality, and started extending it to the functionality described above.

At the moment, what I'm doing is each player has a Camera, their mainCamera, with a PortalCamera script attached. This script creates four RenderTextures: Two to render the view through their own portals, and two to render the view through the other player's portals, i.e one texture for each portal. The PortalCamera script controls the positioning and orientation of two portalCamera Camera objects to see the correct view through each portal.

When running this setup (apart from the hideous loss in fps, which I'll ignore for now), what happens is that one of the players is able to see correctly through all portals, but the second isn't. The reason for this is that the RenderTextures of the portals are set by the player's script, and the other player therefore sees the RenderTexture as generated by the other player's camera.

What I want to do is let the RenderTextures created by player 1's portalcameras only be visible to player 1's main camera, and the RenderTextures created by player 2's portalcameras only be visible to player 2. Essentially, use a culling mask for RenderTextures instead of gameObjects. Is there any way to do this?

Apologies if I violated some question formatting rule, this is my first time posting here.