Make an object move towards another object when button is held then drift when button is released

My most recent conundrum is trying to create a "pull" effect. I'm making a space game where tapping on a small planet will create a line between the player & the planet and the planet pull the player.

The planet will only pull when the player taps and holds on it. What I want to achieve are 2 things:

1) Have the player slowly build up momentum up until a limit (normalize velocity will be used here I believe)

2) Have the player drift at a slower speed when I let go

Right now the speed stays constant but whats really been breaking my head in trying to solve is creating a "drift" effect (like drifting in space). When I let go of the planet the player abruptly stops. I would want them to keep drifting in that direction. If I press a planet in another direction then I want the speed to slow down (not abruptly stop) and quickly start moving to that planet. The effect I'm trying to achieve is Super Mario Galaxys pull stars. https://www.youtube.com/watch?v=eoqZXKQFEng

Here's what I have so far. You can pretty much just throw this on an cube with a rigidbody.

 using UnityEngine;
 using System.Collections;
 
 public class Pull : MonoBehaviour {
 
     public GameObject player;
     public Rigidbody RBplayer;
     public Vector3 Tplayer; //get the players transform
     public bool move;
     public Transform GoTo; 
     public float speed;
     public float maxSpeed;
 
     void Start() {
         player = GameObject.FindWithTag("Player");    //get the player gameobject
         RBplayer = player.GetComponent<Rigidbody>(); //get the players rigidbody
         Tplayer = player.transform.position; //get the players V3
         GoTo = gameObject.transform; //Player to go to current position. So it wont move away on start          
     }
 
     void Update() {
         if(move == true){
             float step = speed * Time.deltaTime;
             //tells the players position by getting the players current position and have it move towards the pully we tap on
             transform.position = Vector3.MoveTowards(transform.position,GoTo.position,step); 
             RBplayer.AddForce(step * GoTo.transform.position);
             RBplayer.velocity = RBplayer.velocity.normalized * maxSpeed;
         }
     }
 
     void OnCollisionEnter(){ //if we collide with any other object then the line between player & pully is broken
         move = false;
     }
 }
 

and here is a seperate script that I attach on the planets that determine if the player should be pulled.

 using UnityEngine;
 using System.Collections;
 
 public class MoveTowards : MonoBehaviour {
 
     public Pull pullScript;
 
     void Start () {
         pullScript = GameObject.FindWithTag("Player").GetComponent<Pull>();
     }
     
     void OnMouseDown () {
         pullScript.GoTo = gameObject.transform;
         pullScript.move = true;
     }
 
     void OnMouseUp () {
         pullScript.move = false;
         pullScript.GoTo = null;
     }
 }

Any help would be greatly appreciated!