Can i raycast against a specific object without using layers?

Just a quick question. I want to cast a ray against a specific target object, that will ignore EVERYTHING else except that target (i'll be casting from an arbitrary point in space)

I know i can accomplish this by temporarily moving the target to a special testing layer, doing the cast, and moving it back. But that does sound a little messy, and (from what i've heard) computationally expensive.

is there any other way to do that before i do the layer thing?