Photon Server Issue

So I made a NetworkCharacter for photon

 private Vector3 correctPosition;
 private Quaternion correctRotation;

 public void OnPhotonSerializeView(PhotonStream stream, PhotonMessageInfo info)
 {
     if (stream.isWriting)
     {
         // We own this player: send the others our data
         stream.SendNext(transform.position);
         stream.SendNext(transform.rotation);
     }
     else
     {
         // Network player, receive data
         correctPosition = (Vector3)stream.ReceiveNext();
         correctRotation = (Quaternion)stream.ReceiveNext();
     }
 }

 void Start()
 {
     correctPosition = this.transform.position;
     correctRotation = this.transform.rotation;
 }

 void Update()
 {
     if(!gameObject.GetComponent<PhotonView>().isMine)
     {
         this.transform.position = correctPosition;
         this.transform.rotation = correctRotation;
     }
 }

and I then attached it to both my Host player and my Client player. Along with photon view (http://i.imgur.com/hSwLAsz.png).

Now the problem is that when I try to move my host everything works fine but when I try to move my client it does not move on both the client and host. If I remove this script then the client is able to move only on his computer and not the host computer but he can see everything that the host is doing.