How would you create an animator controller prefab?

I'm currently working on a 2D top-down game. I'm using an animator controller to animate the Sprite property of my Sprite Renderer on some different characters. I was hoping to create a single animator controller with states like:

• walk_up

• walk_down

• walk_left

• walk_right

Ideally I could apply this controller to all my characters, but swap out the animation clips inside each of the states depending on the character. I assumed I could create a prefab of my animator controller, and replace the clips inside the instances of the animator controller states, however this doesn't work. Is there a better way of doing this so that you don't have to create a bunch of animator controllers with the same states?