Instantiated prefab trying to keep weird Y position

Hey to everybody,

I instantiate a prefab at the location of an empty object serving as a spawn location. Then when I move the prefab to another altitude, it starts acting weird - trying to keep the Y position of the spawn location.

There is my code:

 using System.Collections;
 using System.Collections.Generic;
 using UnityEngine;
 
 public class PlayerSpawn : MonoBehaviour {
     
     public Transform spawnLocation;
     private GameObject player;
 
     // Use this for initialization
     void Start () {
         StartCoroutine (WaitingTime());
     }
     
     // Update is called once per frame
     void Update () {
     }
 
     IEnumerator WaitingTime () {
         yield return new WaitForSeconds (5);
         player = (GameObject)Instantiate (Resources.Load ("Player1"));
         player.transform.position = spawnLocation.transform.position;
     }
 }

The prefab has rigidbody attached and the gravity is checked. It also collides with the floor, meaning both the floor and the prefab have collision on. The weirdness is that when the prefab is brought to another floor height (in my case lower), it tries to keep both its Y position (until it collides with floor) and the spawn location Y position, resulting in flickering behaviour as it can be seen from the video below!

VIDEO

The result wanted is to spawn the prefab at the location of the empty object and then to have its own movement behaviour not relative to the empty object position.

I included my Click To Move script in case it is causing the weird behaviour:

 using System.Collections;
 using System.Collections.Generic;
 using UnityEngine;
 
 public class moveOnMouseClick : MonoBehaviour {
     private Transform myTransform;                // this transform
     private Vector3 destinationPosition;        // The destination Point
     private float destinationDistance;            // The distance between myTransform and destinationPosition
 
     public float moveSpeed;                        // The Speed the character will move
     public Camera camera;
     public LayerMask walkableGround;
     private BuildingPlacement buildingPlacement;
 
 
     void Start () {
         myTransform = transform;                            // sets myTransform to this GameObject.transform
         destinationPosition = myTransform.position;            // prevents myTransform reset
         buildingPlacement = GameObject.Find("Player1Cam").GetComponent<BuildingPlacement>();
         camera = GameObject.Find ("Player1Cam").GetComponent<Camera> ();
     }
 
     void Update () {
 
         // keep track of the distance between this gameObject and destinationPosition
         destinationDistance = Vector3.Distance(destinationPosition, myTransform.position);
 
         if(destinationDistance < .5f){        // To prevent shakin behavior when near destination
             moveSpeed = 0;
         }
         else if(destinationDistance > .5f){            // To Reset Speed to default
             moveSpeed = 30;
         }
 
 
         // Moves the Player if the Left Mouse Button was clicked
         if (Input.GetMouseButtonDown(0) && GUIUtility.hotControl ==0) {
             if (!buildingPlacement.canCancel) {
                 Ray ray = camera.ScreenPointToRay (Input.mousePosition);
                 RaycastHit hit;
                 if (Physics.Raycast (ray, out hit, 500, walkableGround)) {
                     Ray ray1 = ray;
                     float hitdist = 0.0f;
                     Plane playerPlane = new Plane (Vector3.up, myTransform.position);
                     if (playerPlane.Raycast (ray1, out hitdist)) {
                         Vector3 targetPoint = ray1.GetPoint (hitdist);
                         destinationPosition = ray1.GetPoint (hitdist);
                     }
                 }
             }
         }
 
         // To prevent code from running if not needed
         if(destinationDistance > .5f){
             myTransform.position = Vector3.MoveTowards (myTransform.position, destinationPosition, moveSpeed * Time.deltaTime);
         }
 
     }
 }

Any help would be greatly appreciated!

Cheers