OnSerializeNetworkView Sync problem

Here's a demonstration video: https://www.youtube.com/watch?v=Qf84jK-wjWo&feature=youtu.be

Look at front, the second player acts wierd.

Any suggestions?

I'm using the sandard OnSerializeNetworkView method for movement synchronization.

 void OnSerializeNetworkView(BitStream stream, NetworkMessageInfo info) {
         if (stream.isWriting) //this is your information you will send over the network
         {
             sendPosition = this.transform.position;
             sendRotation = this.transform.rotation;
             stream.Serialize(ref isMoving);
             stream.Serialize (ref sendPosition);
             stream.Serialize (ref sendRotation);
         }
         else if(stream.isReading)//this is the information you will recieve over the network
         {
             stream.Serialize (ref realPosition);
             stream.Serialize (ref realRotation);
 
 
         }
     }

And under update

 if(!GetComponent<NetworkView>().isMine){
             transform.position = Vector3.Lerp(this.transform.position, realPosition, 0.1f);
             transform.rotation = Quaternion.Lerp(this.transform.rotation, realRotation, 0.1f);
         }