Physics: Grab Object and Swing Freely Relative to Mouse Cursor

So I have a camera looking down. Lets say I grab an object like a cylinder which is done easily like:

 objectselected.transform.position = mousePos;

Though there is jitter if Im holding onto the object and the camera is flying around (interpolation doesnt help). The objects also tend to flip upside down, is it because their origin point is at the bottom ceneter?

Now I want to add it so that it would swing freely as Im moving my mouse. If I move my mouse in the positive X position it should swing the object so a hard left and if negative a hard right. If the mouse is let go then apply the force relative to the swing.

I suppose you can describe the swinging more realistically as attaching a string to the ball and moving your hang(the mouse) so that it applies the proper force in the direction of the mouse movement.

I have this attached to the gameobject, but its not very effective as it will just have an orbit-like effect.

  GetComponent().rotation = Quaternion.Euler(GetComponent().rotation.eulerAngles + new Vector3(rotationSpeed * Input.GetAxis("Mouse X"), 0f, rotationSpeed * Input.GetAxis("Mouse X")));