Texture memory higher when level loaded from another level?

So I have this level that I deleted all the sprites from because when I load it from another level the profile memory always spikes way high, when I start the level in the editor it says the texture memory is 11.3 mb when I load it from another level that has nothing but one button and a texture memory of 13.3mb it spikes up to 96.6 mb, is this a problem with the profiler or is there something I'm missing about memory management?