How to make a dynamic crowd?

If you have ever played assassins creed you know what I'm talking about. I assume you just create a few animations and a lot of people around the scene, have them just play the animation to look like a crowd.

Here is the problem, like in assassins creed (go to 1:22 in this video) you can walk through them like that. Not including how he pushes them with his hand, would I just put ridgidbodies on the charactures?