Making OnpointerClick event ignore certain colliders.

Hello,

I am using interfaces to implement pointer click and pointer up functionality for some objects in my scene. The game has a first person action camera following my player. The Interface works fine on all the objects except when the player's capsule collider gets in the way.

For example, if an object I want to click is slightly to the left of the player and very close, nothing happens when I click it because the clicks are hitting the player's capsule collider first. I am not saying that the object is obscured by the player, I am saying the player's collider is wider than the player model so it appears I am clicking next to the player character but I am actually hitting his collider. If i move the player a little more to the right I can click the said object.

I've tried putting the player on the Ignore Raycast layer but it doesn't work. How can I get the click event to trigger if my object is masked by another collider?