Group Movement (RTS style) using Unity Pro NavMesh

I have a game in the works that is based off of some maps that I used to play in Age of Empires 2 such as Castle Blood. Using Unity's navigation mesh works great for moving one or two units around. If a group of let's say 20 units is tasked to the same spot, they won't all reach the destination, ever. They'll keep moving and moving and moving forever, trying to push through each other but never succeeding in making it through to the center where the destination is. I have a temporary solution where if the units haven't moved a certain distance in a certain amount of time, it will stop trying to move. What I would like is just for the units to give up on their own without the extra code. This problem also happens if tasked outside the navmesh. They will reach the edge and keep trying to move. Does anyone know of a better way to do this? Note that the destination is only being set once so it's not a problem of the destination being sent repeatedly.