Initial position of an object with interpolation

So I have a basic interpolation system setup for my multiplayer shooter that looks like this:

 private void SyncedMovement()
 {
      syncTime += Time.deltaTime;
      transform.position = Vector3.Lerp (syncStartPosition, syncEndPosition, syncTime / syncDelay);
 
      rigidbody.rotation = Quaternion.Euler (Vector3.Slerp (syncStartRotation, syncEndRotation, syncTime / syncDelay));
 }

 private float lastSynchronizationTime = 0f;
 private float syncDelay = 0f;
 private float syncTime = 0f;
 private Vector3 syncStartPosition = Vector3.zero;
 private Vector3 syncStartRotation = Vector3.zero;
 private Vector3 syncEndPosition = Vector3.zero;
 private Vector3 syncEndRotation = Vector3.zero;
 
 void OnSerializeNetworkView(BitStream stream, NetworkMessageInfo info)
 {
 
     Vector3 syncPosition = Vector3.zero;
     Vector3 syncRotation = Vector3.zero;
 
     if (stream.isWriting)
     {
         syncRotation = rigidbody.rotation.eulerAngles;
         syncPosition = rigidbody.position;
 
         stream.Serialize (ref syncPosition);
         stream.Serialize (ref syncRotation);
     }
     else
     {
         stream.Serialize (ref syncPosition);
         stream.Serialize(ref syncRotation);
 
         syncTime = 0f;
         syncDelay = Time.time -  lastSynchronizationTime;
         lastSynchronizationTime = Time.time;
 
         rigidbody.rotation = Quaternion.Euler (syncRotation);
 
         syncEndPosition = syncPosition;
         syncStartPosition = rigidbody.position;
         syncEndRotation = syncRotation;
         syncStartRotation = rigidbody.rotation.eulerAngles;
     }
 }

My problem is: When a new player connects, the player who was already in the server will "interpolate" from 0,0,0 to his his proper location. So the player who just connected will see him fly through the air in a straight line (super fast) from 0,0,0 to his proper location. Is there any way to make the first player appear immediately at their proper location when a second player connects?

I hope that makes sense, it's a little hard to word it so others can understand.