Gameobjects in Ignore Raycast layer still get hit by raycast.

I have some sphere colliders set as triggers that I need to ignore raycasts. I've managed to set them to the ignore raycast layer through bit shifting. (I know they're there, since Ignore Raycast shows up under layers in the inspector.) But, the raycast still seems to collide with those triggers. We cannot just disable Raycasts Hit Triggers, since we're using NGUI and that needs Raycasts to hit triggers for NGUI's mouse functions to work. Do I have to explicity tell the raycast function to hit all other layers, but the Ignore Raycast layer?