Unity Online Interpolation Lerp Flickering 2D Gameobject

I'm trying to get my basic game online and got into some problems. I followed this tutorial which is really nice and everything worked but one thing, the interpolation.

Without interpolation, I have too much latency issues (the other player is "lagging" a lot). I tried to follow the tutorial, adding

 void FixedUpdate()
 {
     if (networkView.isMine)
     else{
         syncTime += Time.deltaTime;
         rigidbody2D.position = Vector3.Lerp(syncStartPosition, syncEndPosition, syncTime / syncDelay);
 }

And using this method:

 void OnSerializeNetworkView(BitStream stream, NetworkMessageInfo info)
 {
     Vector3 syncPosition = Vector3.zero;
     if (stream.isWriting)
     {
         syncPosition = rigidbody2D.position;
         stream.Serialize(ref syncPosition);
     }
     else
     {
         stream.Serialize(ref syncPosition);
 
         syncTime = 0f;
         syncDelay = Time.time - lastSynchronizationTime;
         lastSynchronizationTime = Time.time;
 
         syncStartPosition = rigidbody2D.position;
         syncEndPosition = syncPosition;
     }
 }

But now the other player is invisible when not moving, and is visible but flickering a lot when moving (flickering so much I hardly see it). Since my method is called from FixedUpdate, I am wondering if the gameobject is simple refreshing so much that I don't even see it. Is that possible?

In any case, would you have an idea on how to fix this?

Thank you very much for your help in advance!

PS: I tried to lerp only if syncTime/syncDelay < 1.0 to be sure I don't extrapolate, but it did not change a thing.

PPS: When I use the interpolation, it flickers a lot but I can see that the position is right, it is not lagging like before, so the best would be to resolve how to make my interpolation and not removing it.