Avatar mask doesn't exclude body part

Hi! I want to use an avatar mask on an animation to get only the upper part of the body (like in stealth tutorial), but when I switch legs off, they keep moving when I start my game. Is that a bug or do I need to do something more?

[I'm on Unity 5 btw.]