Camera culling mask does not work on World Space Canvas

I am trying to cull UI elements selectively using layers but it seems that the camera can't cull anything that is on a canvas. I've looked everywhere and the general answer was that camera culling works for any canvas that isn't set to Screen Space Overlay. I tried my canvas in all 3 modes and none of them work (for my project I need it to be a world space canvas).

If I try to cull an object that isn't UI it works as expected but Unity seems to freak out on UI for some reason. I tried checking the camera's culling mask to see if it changes to the correct mask and it does, I also tried force updating the canvas after setting the culling mask (even though it shouldn't be necessary) but this didn't work either.

I could just turn off the objects but in my project I have thousands of objects and that will be quite the CPU overhaul (I can't do it using threading either because of how Unity works I can't deactivate objects from another thread).

Is this a Unity bug? In which case I'll submit a bug report.

Is there a workaround that won't block the main thread?

Are there any assets out there can could help me?