Select by Layer for a radar system

To my knowledge, there is no fast/efficient way to get all objects on a layer, depending on your definition of fast efficient.

Finding

Find is slow and only finds one object. FindObjectsOfType is very slow.

FindGameObjectsWithTag is really the only fast/efficient option. If you setup your tagging correctly, this is really quite simple. Moreover, since you can have only 32 layers, while you can have n tags, tags are more extensible.

You could loop through the tagged game objects that are returned and only act on ones that match certain criteria.

for(var foo : GameObject in GameObject.FindGameObjectsWithTag("bar"))
    if(foo.layer == 10) doSomething(foo);

But regardless, you'd probably have to do multiple passes as you likely care about several differently tagged objects, so simply tagging properly avoids even having to check layers or any other such nonesense.

Static Caching

Find functions are really only slow and inefficient when called. If you don't call them very often, they're less horrible. If you were to call your Find functions once at the beginning to grab everything, storing only stuff on a given layer into some sort of array, it would only be slow/costly once. If your objects don't change layers, then you would have your array of all objects on a given layer. Otherwise, every time anything changes layers, you'll have to rebuild the array.

Dynamic Caching

You could maintain an array of the things you care about and then use these things in your radar. Add things when they enter the area and remove them when they leave the area. This is more easily done with a System.Collections collection like a List or SortedList.

You could do this with a trigger or collider to represent the radar area. This depends on Unity's messaging system and, if you aren't using the physics system for anything else, this could bring some overhead.

You could do this by a script added to things that show up on the radar that will check their distance to the target whenever their position changes and manage their existence in the radar array. This generally can be optimized to at worst have the cost of a Vector3 subtraction, Vector3.sqrMagnitude and a floating point comparison for every object that has this script every time it is checked.

You could do this by a radar script which raycasts to each of the things you care about up to the radar distance and then adds them to your radar if they were hit. You can layer mask if you don't care about visibility. This requires that each of the objects has a collider.

Alternative ideas

If you are just going to use this for rendering the radar, you could use a radar camera (maybe orthographic) with a layer mask and maybe some replaced shaders or image effects or something.