UNET - Server is spawned in correct spawn position but players are spawned "half way"

I am having trouble with players spawning in the right locations and I've tried creating a spawning system but it isn't working too well. As of now I'm using the Network Lobby Manager to attach player prefabs and once all players are ready, spawn players at game objects with the 'Network Start Position' component.

At the moment the host always spawns at the correct positions while my players spawn near or fairly far away from the correct spawn positions. Wondering if anyone can shed some light on this? (Suspect on scene load spawned objects are still trying to get to correct positions but somehow players are spawning at nearly the same time and touch the spawn point?)

Help is much appreciated (especially from a unity noob)

Thank you!