Help with cameras, player prefab on network

I have a player prefab which is tagged player and has a camera. I would like to have a 1v1 situation with a host / client.

I have been using the built in network stuff and can create and put the two players in the game. But when i create the two player objects they each have a camera and audio listener, and the console complains. Also, I now have two things which are tagged "Player" which confuses some of my UI.

So, how do i have the host and client on separate cameras, where there isnt two cameras in one or both of the players scene?

I tried to make an if statement with the NetworkView.isMine and only tag / activate camera if that's true. I thought it would make it so that it only tags the player and activates thier camera in their own game, but I see on the Hierarchy the host sees both players tagged player and both have an active camera and audio listener.