Orbital cameras and dragging objects with mouse.

I have setup an orbital camera around the scene, and allow the user to drag an object with the mouse. However the object does not always move in the proper direction, depending on the camera position. I have tried using the DragRigidBody.js script instead, but it does not create the results I'm looking for.

 var sensitivity = 25;
 
 private var lastPos : Vector3;
 var pos : Vector3;
 
 function OnMouseDown() {
     rigidbody.isKinematic = false;
     lastPos = Camera.main.ScreenToViewportPoint(Input.mousePosition);
 }
 
 function OnMouseUp () {
     rigidbody.isKinematic = true;
 }
 
 function OnMouseDrag() {
     pos = Camera.main.ScreenToViewportPoint(Input.mousePosition);
     transform.Translate((pos.x - lastPos.x)*sensitivity, 
                         (pos.y - lastPos.y)*sensitivity, 
                         (pos.z - lastPos.z)*sensitivity);
      lastPos = pos;
 }

To clarify: The camera is tethered to the center of the scene at a set distance (with an empty game object) and can be move around by the user (while maintaining that distance). I would like to modify this code to allow the dragging to essentially follow the cursor regardless of where the camera is positioned. The object is weightless and can move in all 3 axis.