Layermask like property for OnPointerEnter/Exit?

I am familiar with raycasting and one of my favorite parts about it is the ability to ignore certain objects through restricting their layer. I have been fooling around with UI elements trying to get drag and drop to work for my inventory system and have made my way to using the OnPointerEnter/Exit functions in order to recognize these elements. I am trying to do this through having OnPointerEnter/Exit functions check to see if the mouse is over an inventory slot when the OnDragEnd function is called, setting that slot as the parent/location for the item object being dragged.

However, when I begin dragging the item object the OnPointer events no longer recognize any sort of collision with the slot below, as the object is in the way, meaning there is no slot for the OnDragEnd function to recognize. Unfortunately, I have a tooltip function that works through a different OnPointEnter script so having the item object ignored entirely is a no-go.

What I need seems to be a local way of ignoring the object in one instance of my OnPointerEnter whilst maintaining it in the other... I am unsure of how to do this and all my inquiries into the matter have been fruitless...