- Home /
How to determine if a specific layer was selected in a LayerMask?
I have seen a number of questions asking "How do layermasks work?" and I have seen a hundred people repeat the same 2 answers over and over but nobody has yet answered the most basic of questions.. at least not in a way that makes any sense to me, that is. :(
All I want to know is: "Is the n'th bit set? Yes or no?"
All I keep seeing is:
This is how you can CREATE a layer mask using bitshifting
Just create a LayerMask variable and select the values in the inspector
Both answers are the complete OPPOSITE of what I am asking. I don't want to know how to CREATE layer masks, I want to know how I can test if a specific layer was selected IN that mask.
You can get the int value of a layer directly from the game object
You can get the layer name from an int
You can get the int value of a layer from a string
...what you can't do is use any of those to see if a specific layer is actually selected in a custom LayerMask variable.
Here, let me explain what I want to do by example:
public LayerMask mask;
bool IsSelected(int layer)
{
//return is the "layer" layer selected in mask?
}
void OnCollisionEnter(Collider other)
{
if (IsSelected(other.gameObject.layer)) DealDamage();
}
All the answers that I have seen tell me how to CREATE layer masks but none of them seem to answer the most basic of questions: How do you test if a specific layer is selected inside the layermask?
Obviously this is not going to work:
if ((mask & 12) > 0)
...since layer 12 would be 100000000000 but "12" is 1100 in binary.
100000000000 & 000000001100 will give you 0.
Some have said "Simply bitshift it right" and that sounded logical until they spoiled it with "and then pass that to a function that works with the layermask". This is where they lost me. If my Layermask is 100011010 and I want to see if layer 3 is selected then I bitshift with 3 and I end up with 000100011 which is 15. How does THAT help me determine if layer 3 was selected?
What I currently do is I get the layer from the object that I hit and I bitshift the layermask right by that number of bits. Now, no matter if the resulting layermask is 000001 or 1111111111, at least I know that the first bit represents the value that WAS at the layer's position... so I test if the first bit is 1 and that is how I check if a certain layer was set or not.
I.e. 1000000000 >> 10 would give me 0000000001 just like 11111111110000000000 would give me 00000000001111111111 but in both cases the 1st bit now represents layer 10.
Thus I do this:
public LayerMask AllowedTargets;
bool IsSelected(int layer) => ( (AllowedTargets.value >> other.gameObject.layer) & 1) == 1;
void OnCollisionEnter(Collider other)
{
if (IsSelected(other.gameObject.layer)) DealDamage();
}
Is this the only way to do it or is there a right way to test "is bit n in the mask set?"
The only thing that really makes this difficult is the fact that Unity will tell you "You hit something on layer 10" and you can't really bitwise AND with the decimal value 10.
If there were a LayerMask.LayerToBinary(int) that could take a 9 and return a decimal function then that would have made life super simple... Hmmm... Interesting idea...
int IntToBinary(int value) => --value < 1 ? 0 : 1 << value;
public LayerMask AllowedTargets;
bool IsSelected(int layer) => AllowedTargets.value & IntToBinary(layer)) == 1;
void OnCollisionEnter(Collider other)
{
if (IsSelected(other.gameObject.layer)) DealDamage();
}
Wonder if that is a viable option?
Answer by Bunny83 · Jun 08, 2018 at 01:16 AM
Well, it's not the only way, though the result will be pretty much the same. Usually you would create a layer mast that only contains your layer you want to test. When you do a bitwise AND operation with your actual mask you only get a 1 if the corresponding bit is set in the mask.
int yourLayerMask;
int layerIndex; // index of layer you want to test:
if (yourLayerMask & (1 << layerIndex) != 0)
This will return true when our bit is set in the mask.
// 0000000101010110
// 0000000100000000 // layer index 8 --> (1 << 8)
// & // bitwise AND
// 0000000100000000 // result will only have a 1 at our bit position if the bit is set in both operands of the AND
This is how bit comparisons work in general. You can also test several bits at the same time as long as you interpret the result correctly. Imagine a given value A:
A = 010001010110
Now imagine you want to make sure certain bits are set in A. Lets define them in B:
B = 000000011111
Now when you do C = A & B
the result will be
C = 000000010110
So here only some bits are set. You can test if at least one of the bits in B are also in A by doing:
if ((A&B) != 0)
You can also test of all bits in B are set in A by doing
if ((A&B) == B)
if (yourLayer$$anonymous$$ask & (1 << layerIndex) != 0)
I think that is an elegant way to do it, yes. I shift the entire Layer$$anonymous$$ask right then check the first bit whereas that takes the object I hit and bitshifts it left for a clean "int & int > 0" test.
Boils down to the same thing but your way feels cleaner / neater / tidier / less of a hack.
Your entire reply focuses on binary values and I was specifically hoping there was a way to skip the binary math in deter$$anonymous$$ing if a layer is inside something that is just a container for layers... but at least with $$anonymous$$imal knowledge of binary the math is simple enough that one can live with it, I suppose. :(
Thanks for pointing out how I can make my code that little bit cleaner. :) I did ask for "a better way", after all :)
Your answer
Follow this Question
Related Questions
Using a bitwise operator with LayerMask 2 Answers
LayerMask, DrawRay and DrawLine acting weird... 1 Answer
Assigning found enemy to target variable 2 Answers
layers not ignored by raycast 1 Answer