Only interact with objects if they're visible and unobstructed by other objects?

So let's say I have in-game panels or windows. I can use buttons and box colliders and things to have close buttons or areas that allow you to drag the panels.

If I drag a panel so it's over the top of another panel (I use layers so the active panel is always in front), I'm still able to click the buttons in the lower panels. I can close a panel by covering the panel with another and remembering where the button is.

I'm fairly new and making these experiments to figure things out but I'm wondering what an optimal solution would look like. My current thinking is allowing the sprite rendering of the background panels but disabling all their interactive components. Or having all the components on a child and disabling the child. Is there something Unity allows that's better than that? Or some way to disable all children of a parent or all components of an object without doing them one by one? I guess I could script a for loop for children but there might be something better.

Tl;dr if the player can't see the object because there's something else in the way, I don't want them to be able to interact with it. What technique would you recommend for getting me started on this?