Moving a player character that has ragdoll colliders/rigidbodies

I have a player character that I'm moving with a character controller script and it is moved by applying velocity to a ridigbody.

This works when I have a rigidbody component on the player but my character also has ragdoll colliders and rigidbodies on each body parts (set up with Untiy's ragdoll wizard).

What I'd like to do is remove the ridigbody and collider that are on the player and use the ragdoll colliders and rigidbodies instead for physics movement and collision.

But it's not working! When I use the rigidbody from the skeleton Root and try to apply velocity the character won't move (animation plays but she keeps running on the same spot). What do I need to do to get this working?

Another issue is that if I leave my rigidbody on but remove the large capsule collider I can move the character but collision doesn't work and she runs through static objects. I already tried with convex set to true on static meshes but it doesn't make a difference.

Does somebody know what causes movement and collision not to work with the ragdoll rigidbodies and colliders?