Optimizing for multiple objects in one blend file

In my game players are able to build a building with rooms. As more adjacent rooms are added, certain overlapping walls/ceilings/floors become hidden (deactivated). My current setup is one blender file that contains all the objects so that I can see if the scaling/overlapping/texture mapping correctly blend into each other, however in Unity when this is imported my prefab contains an hierarchy of all these objects. All these objects shares the same texture.

I am planning to release the game for mobile devices with very limited memory and I was wondering that if I have many instances of these prefabs in my scene but most objects are inactive (hidden), will this waste a lot of memory on a mobile device or will Unity optimize the memory usage, CPU & GPU calls for these inactive objects? Or will I be better off to split up these objects into multiple separate blender files and import it individually into different prefabs which I switch (unload/load) as the building changes?