Manual Timeline with RigBuilder

So I have my timeline running on manual mode since I need to be able to play the timeline at different speeds.

As soon as I add the RigBuilder component the weights on the Rig becomes blue/red indicating it's being controlled by the timeline/animator even so there are no keyframes on the weight properties.

I saw you can call Evaluate on the RigBuilder.graph so I do this when I evaluate the main timeline in LateUpdate, but before or after doesn't make any difference or change.

This can be recreated by making a Timeline effecting and Animator RigBuilder combo, setting the timeline to manual and Evaluating the delta time of the PlayableDirector in LateUpdate.

I need the Rig to behave the same as when using GameTime.