Is there some way to combine continuously scrolling backgrounds AND 2D Physics in tandem?

I'm creating a 2D game where I'd love to take advantage of the Unity Physics engine, but I want to take on a large scale by continuously tiling the background and generating obstacles and enemies on the fly as the player hurdles through the air (with limited control over their movement). Does anyone have any hints as to how I might handle this in a memory-efficient manner?

So far I've considered building a script that predictively renders backgrounds tiles and other objects based on the relative direction of the player, but I'm not sure how smart Unity is about cleaning up the "left-behind" game objects. Would I need to implement my own garbage collection or is Unity good about clearing memory used by gameObjects that live outside the viewport?

Also, is this ill advised since all the player movement will be global (as opposed to relative), causing the player coordinates to potentially grow outside the bounds of a float? I don't know of any way to both use the physics engine and also handle movement in a practical and relative fashion, and I certainly wasn't hoping to make my own physics engine from scratch.

I'm not looking for anyone to spend time building code for me. I'm just looking for a conceptual basis to accomoplish what I want to do. For the sake of reference, I want to make a game like the Newgrounds classic "Toss the Turtle."

Thanks so much!