Unity Scene "remembered" even after restore

Really hard to describe but I backed up all my scenes, I was messing around editing a scene knowing I had a working backup so was toying around moving stuff about and generally seeing what models and prefabs I had. After I messed about I accidentally saved it so I replaced the scene in its folder and thought that would just restore it back to its original condition. Wrong, It seems to have remembered everything I did even though I replaced the scene. I reimported it and re-imported "all" and after that was finished the scene is still broken and I have officially broke my project. What am I doing wrong? I thought backing the scenes up would save me from any mistakes I may do but it seems not.