Mouse follow on network

Hi, I'm having trouble when trying to make the player's object to follow the mouse pointer.

With any other kind of control, like the directional keys, everything works fine, but there's something wrong when I try the mouse following method.

It seems that all Game Objects (with the mouse follow script) in the scene follow all players' mouse positions in the network. So, the mouse position is not individual.

This is the current mouse follow script I'm using:

 if(networkView.isMine)
      {
      mousePos = Input.mousePosition;
      var ray : Ray;
      ray = Camera.main.ScreenPointToRay(mousePos);
      var hit : RaycastHit;
      
      if(Physics.Raycast(ray, hit))
      {
                transform.position = Vector3.Lerp(transform.position, hit.point, 0.5   * Time.deltaTime);
                transform.LookAt(Vector3(hit.point.x, 0, hit.point.z));
     }
 }