OnMouseDrag() Problems C# script

Ok so I am trying to make a game with a perspective camera that is at an angle to the game play world. In this game there are objects which need to be selected by the player and then dragged into position to act as barriers. I did a quick search and found a similar question and tried to implement the code below.

 void OnMouseEnter()
 {
     if(Input.GetMouseButtonDown(0)){
         screenPoint = Camera.main.WorldToScreenPoint(scanPos);
  
         offset = scanPos - Camera.main.ScreenToWorldPoint(
             new Vector3(Input.mousePosition.x, Input.mousePosition.y, screenPoint.z));
         isMoving = true;
     }
     if(Input.GetMouseButtonUp(0)){
         isMoving = false;
     }
 }
  
 void OnMouseDrag()
 {
     Vector3 curScreenPoint = new Vector3(Input.mousePosition.x, Input.mousePosition.y, screenPoint.z);
  
     Vector3 curPosition = Camera.main.ScreenToWorldPoint(curScreenPoint) + offset;
     transform.position = curPosition;
 }

I had to modify it, because in my game world Y is up, so that the object wouldn't try to get some love from the camera. So I fixed the Y to keep the object on the ground which was working perfectly, but the object would still move to the camera's position and just stay on the ground, unless it felt like moving to some other random place, because it did that at times too. However here is the thing that really made me go WTF? If I selected the object in the scene view or the heirarcy and then tried to click and drag it in the game view it would move up to the fixed Y position in the spawn location, and not move again.

Then I had an idea to try and move the object by just taking in the difference of the mouse position and adding that to the transform.position of the object, and so I modified the code to look like this.

 void OnMouseEnter()
     {
         currMousePos = new Vector3(Input.mousePosition.x, 1, Input.mousePosition.y);
         lastMousePos = currMousePos;
         if(Input.GetMouseButtonDown(0))
         {
             isMoving = true;
         }
     }
 
     void OnMouseDrag()
     {
         while (isMoving)
         {
             currMousePos = new Vector3(Input.mousePosition.x, 1, Input.mousePosition.y);
             Vector3 curPosition = currMousePos - lastMousePos;
             transform.position += curPosition;
             lastMousePos = currMousePos;
             if (Input.GetMouseButtonUp(0))
             {
                 isMoving = false;
             }
         }
     }

Which does absolutely nothing, because now the object just sits there at the spawn location. Please help me.