Issuing SetClientReady causes duplicate network objects on local client

I am writing a client server game with UNet. I have created a working framework originally using NetworkManager class as a basis. I am now attempting to rework it without using NetworkManager.

At this point I have it mostly working, in that clients can connect to the host and pass messages back and forth reliably.

However, the local client is able to see the Hosts objects in the game scene. These objects are all spawned via NetworkServer.Spawn calls. This really becomes a problem when I issue a NetworkServer.SetClientReady message for the local client as it causes the local client to then spawn a local set of object instances. These obviously interfere with the ones on the host leading to some odd behaviour :)

Remote clients don't suffer this issue obviously and behave exactly as you would expect.

I am wondering if there is something I should be doing when spawning an object that is for the host, so that it doesn't appear on the local client's gamescreen?

Code snippet follows, this is all from my gameManager class. The hdl_ClientReady is called once my client sends a custom message to say that it is ready.

 void Start() {
         ClientScene.RegisterPrefab(tmpPlayerPrefab);
         netConfig = new ConnectionConfig();
         netConfig.AddChannel(QosType.Reliable);
 }
 
 void StartServer() {
         NetworkServer.Configure(netConfig,10);
         NetworkServer.Listen(7777);
 }
 
 private void hdl_ClientReady(NetworkMessage msg) {
 NetworkServer.SetClientReady(msg.conn);
 }
 
 void OnClientConnect(NetworkConnection c ){
   //....
   NetworkServer.Spawn(n);
 }