Network | Math: Smoothly lerp between inconsistent intervals

So i'm working on a 1v1 2D game (using rigidbody physics + forces to jump/move), i send position+velocity data once every 0.05 seconds IF the distance between the last sent position and the current position is larger than a set amount. So data arrives at inconsistent intervals and lerping causes a jittering motion. Is there anyone with a large enough brain to help me figure out how to smoothly lerp the position of the opponent that's received at inconsistent intervals? (e.g. intervals of: 0.06, 0.13, 0.83, 1.64, 0.67, 0.32 etc) I'm using C# but an explanation of the logic for the solution would be good enough!