How does the unity terrain shader handle an arbitrary amount of textures ?

Multiple passes. If you get hold of the terrain shader source code, you find two shader files associated with it - one labelled "FirstPass" and one labelled "AddPass". When you have 5 or more textures, Unity draws the first four in one pass, and then renders the whole terrain again with a second pass, allowing for 4 more textures. If you need more than 8 textures, then it needs a third pass, etc.

This is why having more than 4 textures gives you a significant performance drop - you are essentially drawing the entire terrain twice.