Camera problems when reloading scene

I have a single-scene endless runner game, where when you die you press restart button and the scene reloads itself. The problem is that when I reload the scene, the camera doesn't "spawn" to the exact same position as when you start the game, but shifts to the right. I think why this happens is because the scene takes too long to load and so camera lags behind. Have you had similar problem or know how to deal with it ? By the way my game is running at constant 60fps and I also have a camera pan animation when you press play button. When start button is pressed: When restart button is pressed: