Animation rigging and animation layers not working together

Hello, I have been researching for around 8 hours on animation rigging and all its' uses. Very handy for everything except when using animation layers as it seems. I have a base layer, but then I have an override layer for the upper body to allow for melee attacks while running etc. Now I have figured out how to bake motion to constraints for certain animations, and so my wish was to be able to play a running animation in the base layer and then override the upper body with an attack animation which should animate the constraints resulting in a correct rotation of the spine in object space independent of the lower body motion but it appears it entirely ignores to run the animation for the constraints as long as that animation is not played in the base layer. Have I missed something or is this just not possible?