Giant colliders restricted to a set size?

I am working on a product for a client and we had to scale up our units so that 1 unit = 1 inch (since the client is using CAD models for their product and 1 inch = 1 unit).

Now the problem we are having is when we scale up the colliders (around 50x50x50 units), we have all sorts of different shaped walls that have mesh colliders on them (because they can be curved and changed). Whats going on is when someone drags the collider object into the wall it does not get detected at all. Now once the collider center gets within 10 units (yes, I did all sorts of testing and it turned out to be 10 units) it finally gets detected.

I feel like this is an internal problem and its not something I can fix or change.

And yes, instead of mesh colliders I did use box colliders and it did the same thing.

Thank you for reading,

Nick Drexl