Can physic2d linecast and overlapcircle do a layer priority?

I have a problem with finding target mechanic for my game.

The units has 3 type: organic, non-organic, and aircraft

Some units have a priority target based on the type for example RPG-Troop must target non-organic first before looking for organic unit. I used linecast for for player target detection since player units can only attack enemy in first floor and overlapcircle for enemy, since they can attack from second floor.

The problem is, are raycasting(linecast and overlapcircle) have something to put layer priority? or am I forced to use linecastnonalloc and overlapcirclenonalloc to do this? If I don't have alternatives except using linecastnonalloc and overlapcirclenonalloc, will it make the game lagging since considering the level will have many units at once?