Endless "Falling in place" Simulation Question

Hello,

I searched the forums and the internet in general but can't seem to find an answer to this so I thought I would ask:

I am trying to make a small "endless runner" type game and I want the player(currently a sphere) to fall continuously. I am pretty sure I can't just loosely attach a camera to the player and just add a rigidbody because it will just keep going further and further away from the origin, and as far as I've heard that would cause problems.

Ideally I would create an object pool and scrolling background with another camera and just have the player only be able to control their movement on the X axis. But if the player were to hit something on the way down (part of the design, they have to dodge) they get moved to the top of the screen and stay there with the controls I have now( I don't have a rigid body because of the above reason) for a penalty. They should come back to the center of the screen(after a certain amount of time) to continue falling assuming they didn't go off the top and out of view(game over).

The problem is if the player were to hit something that is supposed to make them go sideways, I would want them to bounce to the left or right and have it seem like an actual bounce and not a slight "nudge". That way I could also just use a Physical material. Unfortunately without gravity I don't know how I would achieve this effect.

So I am basically asking how one would go about simulating gravity so the player would stay inside the camera view and it looks like they are falling and bouncing without the camera actually moving with the player and the player doesn't go far either?

Im sorry if this isn't the best place to ask this and if so please tell me. And I'm not expecting a detailed answer just if you can point me in the right direction.