Assigning a texture for object in inspector. Is it loaded right away?

Sorry that I couldn't find a clearer way to write the title. The question I have is the following...

I have a game for mobile where lots of different furniture will be instantiated at runtime. However, the furniture are skinnable - the user can create texture files in a specific folder to change the looks of the object when playing. To address this situation, I have each possible furniture to hold a reference to its original texture (a Texture2D reference in the script, receiving a Texture2D from my assets folder) and a file name in case of customized textures.

Since there are lots of possible different furniture, and the user can skin their textures, I made a script that automatically creates a texture atlas of the most frequent textures being used to reduce draw calls. However, in this step, I will need many of the furniture objects on memory. Even if I dont assign the textures in a material, will the Inspector reference result in all different textures referenced from my assets folder being loaded into memory? That's a huge memory footprint. If this is true, is there a way to load the texture from a reference only when needed, without needing to put them into a Resources folder?

Thx