Expense of CharacterController.Move

Hey everyone! So I started messing around with a new prototype where the players different inputs control multiple different game objects in the scene. Since these objects are almost always "characters" I figured that using the CharacterController component made perfect sense. I don't believe I have ever made anything that uses this component more then once before this. It seems like the call to Move or SimpleMove is EXTREMELY expensive. I am looking at a giant decrease in framerate when I am moving more then one object in the scene. Are these calls really this expensive or am I missing something? If I need to move multiple characters in a scene is it generally considered to be good practice to not use a CharacterController component?

I can post code when I am back at home later I just figured I would type up the question while it was fresh in my mind. Thanks!