[UNET] Clients position not being synced after being replaced with ReplacePlayerForConnection

Hello Community!

I've recently started on a simple multiplayer project to learn the ropes of UNET. I got a general idea of how everything works now, but I've stumpled upon a problem when trying to use NetworkServer.ReplacePlayerForConnection.

Thus far, I've got a NetworkManager that spawns a PlayerController (A lobby'ish kinda thing) that controls which team the player should join and setting the player name:

     public override void OnStartLocalPlayer()
     {
         if (!isServer) NetworkServer.Listen(NetworkServer.listenPort);
 
         base.OnStartLocalPlayer();
 
         gameManager = GameObject.Find("_GameManager").GetComponent<GameManager>();
 
         SetUI("Prematch");
 
         CmdSetPlayerName(gameObject, "Player_" + netId.Value.ToString());
     }

After joining the server, the playerName is being set through the server and then synchronized via SyncVar.

Now, the player has but choice and that is to join the game through a button. On click, this runs a method that sets the team and tells the server to create the player object with the NetworkServer.ReplacePlayerForConnection:

 void Btn_JoinGame()
     {
         if (gameManager.team1Players.Count <= 0)
         {
             CmdSetTeam(playerName + "_Controller", Team.One);
         }
         else
         {
             if (gameManager.team1Players.Count <= gameManager.team2Players.Count)
             {
                 CmdSetTeam(playerName + "_Controller", Team.One);
             }
             else
             {
                 CmdSetTeam(playerName + "_Controller", Team.Two);
             }
         }
 
         CmdAddPlayer(playerName + "_Controller");
 
         SetUI("Game");
     }

Now, the CmdAddPlayer does the following:

     [Command]
     void CmdAddPlayer(string target)
     {
         PlayerController pCtrl = GameObject.Find(target).GetComponent<PlayerController>();
 
         GameObject p = Instantiate(playerPrefab);
         p.GetComponent<Player>().PlayerName = pCtrl.playerName;
         p.GetComponent<Player>().Team = pCtrl.team;
 
         NetworkServer.ReplacePlayerForConnection(pCtrl.connectionToClient, p, 0);
         NetworkServer.Destroy(pCtrl.gameObject);
     }

It all works quite well actually, the server instantiates the player on both the client and the server, and both players can move around and send messages to each other, but the clients position on the server is now not getting synchronized. I'm using the Transform Sync thingy that comes with UNET.

Any help is greatly appreciated!