Prefab Materials and Meshes being deleted, How can I stop this

Background, this is for data driven loads so I can deter$$anonymous$$e at runtime which prefab I want to create. Second I want a pool of gameobjects to draw from because I actually am loading say 3 different prefabs and when I need a gameobject I am getting one from the pool and then setting its mesh/material to one of the 3 prefabs I have loaded. This gives dynamic loading without hopefully large memory overhead.

But currently has the drawback of screwing up the actual real prefab.

example debris, i want a pool of debris gameobjects, so I create 100. But I don't want the debris to be the same display. So I load 3 gameobjects with textures of debris. Thus procedurally I deter$$anonymous$$e which of the 3 textures I need to use, and assign it to the debris pool item I have received. I should have a highwater mark for memory with the 100 pool items and the 3 differences.

(just picked numbers out for the example, I have much high numbers intended and highwater marking the memory usage is critical)

Just out of curiosity are you creating a new material when updating A's materials or simply updating the current material it uses?