Problem with rendering after using localScale

So I'm making a little sandbox game and I'm having issues with objects that I use localScale on.

The scaling itself is not the issue, however if I assign different materials to the object after it's been scaled, the object doesn't render properly anymore. When looking more to the outside edges of the object, it simply vanishes. Pretty much when the un-scaled version of the object would not be in view anymore, it disappears even though the scaled version is still in plain view. This ONLY happens after I change materials or sharedMaterials.

Here is my script:

 if(control == true)
         {
             editMode = 0;
             Material[] mats = new Material[2];
             mats[0] = HoldingObjectRenderer.sharedMaterial;
             mats[1] = HighlightMaterial;
             HoldingObjectRenderer.sharedMaterials = mats;
             holdingObjectActive = true;
         }
         else
         {
             holdingObjectActive = false;
             holdingObject = null;
             server.HandleObject(obj, false);//Let server know to stop controlling the object
             
             Material[] mats = new Material[1];
             mats[0] = HoldingObjectRenderer.sharedMaterials[0];
             HoldingObjectRenderer.sharedMaterials = mats;
         }

Any help is much appreciated :)