How do I replace parts of an animated character?

We've created a humanoid character for a fantasy RPG. The character model itself is seperated into modular pieces (head, upperbody, legs, hands, feet). The reason for this is that we want to replace these modules when the player is equiping items (e.g. replacing the hands with a glove model or replacing the upperbody with an armor model).

However after trying to replace a module we notice that the animations are either a few frames off or not working at all. Is there anything special we need to be aware of? We've been trying to fix this issue for quite some time now, trying different ways always ending up with the same problem.

The character and armor models have been created and rigged using Blender and were brought into Unity as seperate files. We're using Mecanims humanoid rig type to animate the character. Character and armor models are working well as long as we keep them seperated.