Player GameObject is not being positioned where my SceneController tells it to after scene transition.

My game has two scenes. One is called TownScene and the other CaveScene. The Player can navigate between these two after he passes through triggers, one located at the entrance of the cave and one at the exit of the Cave. I wished my Player would spawn close to the entrance/exit of the cave to get that good felling of continuity for the game, and I'm trying to do that through code, but failing miserably.

My player has one script with a Singleton, so that the same instance of the object is kept between scenes.

I have a SceneContoller script that exists in both scenes. It has two Serialized fields that hold the exact coordinantes I wish my Player would spawn. Here is the script:

 public class SceneController : MonoBehaviour
 {
     [SerializeField] Vector2 townSpawnPoint;
     [SerializeField] Vector2 caveSpawnPoint;
     [SerializeField] GameObject playerGameObject;
 
     //private GameObject playerGameObject;
     private string activeSceneName;
 
     private void OnEnable()
     {
         SceneManager.sceneLoaded += OnSceneLoaded;
     }
 
     private void OnSceneLoaded(Scene scene, LoadSceneMode mode)
     {
         GetActiveScene();
         if (activeSceneName == "TownScene")
         {
            playerGameObject.GetComponent<Rigidbody2D>().transform.position = townSpawnPoint;
         }
         if (activeSceneName == "CaveScene")
         {
             playerGameObject.GetComponent<Rigidbody2D>().transform.position = caveSpawnPoint;
         }
     }
 
     private void OnDisable()
     {
         SceneManager.sceneLoaded -= OnSceneLoaded;
     }
 
     private void GetActiveScene()
     {
         activeSceneName = SceneManager.GetActiveScene().name;
     }
 
     public void LoadCaveScene()
     {
         SceneManager.LoadScene("CaveScene");
     }
 
     public void LoadTownScene()
     {
         SceneManager.LoadScene("TownScene");
     }
 }

You can see that the OnSceneLoaded method will pass the spawn coordinates to the Player's Rigidbody right when the scene is enabled. That works fine, but just once, when I first play the game. That means that starting the game in the TownScene will make the Player spawn at the townScenePoint, and that starting the game in the CaveScene will make the Player spawn at the caveScenePoint. But after that, if I go through the triggers, the Scene will switch and the Player will keep the coordinates from the previous scene.

I cannot figure out how to fix this. I did some Logs to see if these methods were being executed and they are. So how can I make it run consistently? I would appreciate some help, I have been trying to fix it for a couple of days with no luck.

Another issue going on is that, in the very begging of the game, I wished my Player would spawn at the town entrance, but as I said above, the SceneController script will make him spawn at the townSpawnPoint (that's at the cave entrance). I could not get around that either, but I would be glad already if I could fix the first issue. Sorry for the long text, but I tried to be very clear about what is going on.