Drag handler is blocking image IPointerClick but not button OnClick

I have a very complex UI with lots of menus and windows for my RPG.

On most of my major windows I have IBeginDragHandler and IDragHandler handler to let the player move the windows around. I also use lots of buttons and OnClick events. recently I'm finishing up a window I started a long time ago that is using images where normally I would just use buttons now, each with a script handling the child text on it and its IPointerClickHandler.

So on this window I have buttons with OnClick, and images with IPointerClickHandler. These are children of this window and within it's 500x300 RectTransform.size, but it is only a Transform; there are no panels or anything covering the images or buttons.

Ever since I re-enabled the IBeginDragHandler and IDragHandler on this window (again, just a Transform with a manager script on it) the images are no longer responding to clicks, but the buttons are responding as normal. If I turn off the drag handlers the images start to respond again.

Can someone explain this?