I am trying to allow the player to pick up and move objects in a FPS style game

Basically what I am trying to achieve is an effect similar to using the gravity gun in Half life 2...

I have the player from which there is a raycast to the object on mouse click. Once the raycast detects collision with a rigid body the effect should take place...

The player should be able to move the mouse around the screen and the object should follow the mouse movements. The player should also be able to walk around and maintain the same distance from the object being held.

here is my subroutine that is run While the mouse button is being held.

NOTE: this actually works somewhat, how ever moving the mouse to quickly will drop the object, also my math is off so the object gradually gets further away.

IEnumerator fireLaser() { beam.enabled = true; while(Input.GetButton("Fire1")) {

 Ray ray = new Ray(transform.position, transform.forward);
 RaycastHit hit;
         
 beam.SetPosition(0, ray.origin);
         
 if(Physics.Raycast(ray, out hit,100))
 {
     beam.SetPosition(1, hit.point);
     if(hit.rigidbody)
     {
                     //subtract 1/2 object size from hit location to get object center

         Vector3 tempScale = hit.transform.localScale;
         tempScale = tempScale/2;
         Vector3 manipulatedHitPoint = hit.point;
         manipulatedHitPoint = manipulatedHitPoint + tempScale;        
     
         hit.transform.position = manipulatedHitPoint;

     }
 }
 else
 {
     beam.SetPosition(1, ray.GetPoint(100));
 }
 yield return null;
 }//end while

}//end CoRoutine

Please help, This is my first attempt at developing a game, I have a couple years coding experience but virtual none with unity or C#