Collision Matrix not working properly

I looked around a bit but couldn't figure out what was up. Here's my setup:

I have 2 GameObjects, A and B, each have a different layer, layer LA and layer LB. Each of those GOs have a child, C and D, each child are on different layers as well, layer LC and LD, where LC is child of LA and LD is child of LB.

My matrix only allows interaction between LA and LD, LB and LC. Each of these objects have their own colliders which are triggers.

Here's the problem: In theory OnTriggerEnter/Exit wouldn't be called between say LA and LB or LA and LC, etc. My problem is that LA and LB are triggering only LA side, which shouldn't be happening. I have an easy workaround for this in my code (a null check fixes it) but I'm pretty sure that isn't supposed to happen (or I have no idea what's the Collision Matrix's point).

Is that a bug or am I doing something really wrong?

And I'm 100% sure it's A and B colliders that are triggering A's OnTriggerEnter, it's not anyone else's collider.