How to select different player prefab for every player (multiplayer)

Hi I'm making my 2nd game and i'm still learning . This is probably a stupid question but how do i select different player prefab for every player? I have a custom Network Lobby Manager with this code :

     public GameObject 1Pref;
     public GameObject 2Pref;
     public GameObject 3Pref;
     public GameObject 4Pref;
 
     public NetworkLobbyManager lobbyMan;
 
 
     void Start () {
         lobbyMan = this.gameObject.GetComponent<NetworkLobbyManager>();
         CharacterSelect ();
     }
 
     void CharacterSelect () {
         //Searching to find which player was selected
         if (characterSelector.character == null) {
             lobbyMan.gamePlayerPrefab = 1Pref;
         }
         if (characterSelector.character == "1Pref") {
             lobbyMan.gamePlayerPrefab = 1Pref;
         }
         if (characterSelector.character == "2Pref") {
             lobbyMan.gamePlayerPrefab = 2Pref;
         }
         if (characterSelector.character == "3Pref") {
             lobbyMan.gamePlayerPrefab = 3Pref;
         }
         if (characterSelector.character == "4Pref") {
             lobbyMan.gamePlayerPrefab = 4Pref;
         }
     }

but this doesn't work probably because i have a lobby with minimum 2 players instead of a network manager so : the playerPrefab is changed for player1 -> the playerPrefab is changed for player2 -> players are spawned according to the current playerPrefab entry which is the playerPrefab for player2 and as a result player1 has been spawned with the same playerPrefab as player2 instead of the one chosen . Am i missing something?