koobas.hobune.stream
LayerMask vs 1<< gameObject.layer ?
I need to do a raycast on a specific layer. I thought I'd use gameObject.layer and build the mask myself, but I don't get the same value as if I use a LayerMask and pick the layer manually. Ideally I'd prefer using the object's layer than having to add another inspector member.
I couldn't find any explanation for my results. According to the documentation, 1 << layer should work. Any idea? Thanks!
EDIT: Here's some additional information as requested by, erm, FriskyDingo.
int raycastLayerMask = 1 << gameObject.layer;
Physics.Raycast(..., raycastLayerMask);
gameObject.layer is 15, which corresponds to the correct layer number in the layer settings. Using this method, 1 << 15 is 32768. However, using a LayerMask, the int value when I check layer 15 is 1024. 2^10 is 1024.
The thing is, the bit shifting results seem more logical to me, but the LayerMask seems to be right. My raycast hits the expected object when layerMask is 1024, but not 32768.
Where does that come from? I have no idea. There doesn't seem to be any relation with the built-in layers either.
So there. I'm stumped! :)
Can you include the offending code, the results you're getting and the results you expected?
The problem is that humans think of layers as 0, 1, 2, 3, 4 but computers think of them as 1, 2, 4, 8, 16 (in binary, 1, 10, 100, 1000, 10000.) You just need to keep straight whether it expects a human layer or a computer layer:
o gameObject.layer uses human numbers. When you set gameObject.layer=10 and print, it tells you 10, but it internally sets layer to 1024 (in other words, the GET and SET for layer are doing a lot of work.)
o Raycast uses computer numbers, for speed. Both of these check against layer 2 (Ignore Raycast)
RayCast(.... (1<<2));
RayCast(.... 4);
When you use a human number where it expects a computer one, or vise-versa, randomish things happen:
// ERROR:
RayCast(.... thingToCheck.layer); // if layer is 12 (1100) you are really doing a
// raycast on layers 2 and 3
// WORKS! convert from human 1,2,3 back to computer 2,4,8:
RayCast(.... (1<<thingToCheck.layer));
I'd say if you want to use gameObject.layer directly for raycast masks, best thing to do is precompute the real one:
int catLyrMask = (1<<someCat.layer); // in Start?
RayCast(.....catLyrMask);
That's pretty much how I understood the issue. Raycast takes a mask, while GameObject.layer is a simple number, hence the need to bit shift. I edit the question with more details. I'll keep looking at it, maybe there's something else wrong in my code.
I found the problem in my code. I still gave your answer a thumbs up because it's a pretty damn thorough explanation of masks that will apply to more people than my own answer will. Thanks!
I ran across something similar and posted it here (with resulting function to handle the test) http://answers.unity3d.com/questions/150690/using-a-bitwise-operator-with-layermask.html?viewedQuestions=122586&viewedQuestions=150690
Well that's embarassing. Yet again! Anyway, the thing is, I'm trying to use the object's layer as if it were the whole layer collision matrix. Of course it doesn't work. I'm telling Physics.Raycast() to collide against the object from which I'm casting the ray, rather than using the collision matrix associated with its layer.
As there's no way (that I know of) to get the collision matrix from a script, I guess I'll have to leave the LayerMask exposed.
Thanks for your help!
