LayerMask Not working when raycasting to Specific layer.

Hi all, I'm having a problem with my raycast where it feels like its ignoring the layermask that i give it. Despite being told to only hit the "Floor" layer, the Default and Player layers are also getting hit regardless.

Heres the code where the raycast is done, as you can see i've specified the layermask and put it in to the raycast. The code makes it so the material of an object is changed when hovered over. Despite the layermask, any object i hover over has its material changed, not just the floor. Any help would be appreciated, thanks.