Get reference to spawned object from server on client

I don't have a multiplayer game. I'm using my laptop as a server, and a Hololens as my client.

I've spawned a prefab on the server with client authority to the hololens, and it spawns correctly. I tried adding a MsgType.ObjectSpawned, but it is never called on the client side, so I have no reference to the spawned gameobject on my client. Essentially, that's all I'm trying to do, get a reference to the spawned gameobject on the client. I have three seperate gameobjects in my scene: a client, a server, and a clientServerManager. The clientservermanager makes whichever one active depending on the machine it's run on. This is because most of the code is different for both the server, and the client. The only thing they really need to know are the same classes/prefabs, and spawned objects.

A little bit about the actual thing I'm doing it for:

Hololens sends device position and spatial map mesh data in world space to server, which visualises this in the game view. The server can spawn objects wherever the hololens is looking towards, which works. I've added a directional indicator to the spawned object, so the user on the hololens can find it. Unfortunately, I added it the spawned object prefab, so it adds one for each new one. I want to have it on just the latest.

So I added my directional indicator script to the client, and make a currentObject GameObject reference. I want to set the latest spawned object to this currentObject.

Again, getting the reference will be useful for other things, but this is the specific use case I'm doing it for now.

Thanks!

EDIT:

Oh, also, I'm not using a NetworkManager. I'm just using NetworkServer on server, and a NetworkClient object on client.