Working with Normal Maps and Sprite Atlases

All my sprites for my character are separate and put into a sprite atlas, but I found out (the hard way) that when I put a normal map on those sprites (as secondary textures in sprite editor), they don't work because of the sprite atlas (they work fine after taking the sprites out of the atlas).

So what's the best practice here?

I've been setting my normal maps to normal map when I import them, but that means I can't make an atlas of them. They seem to still work when set to sprite, and I'm guessing if I do that and make an identical atlas to the other sprites it will work. But there must be a reason there's a normal map mode, so should I be using it?

The only other option seems to be to not have anything that needs a normal map in an atlas, but that would be a huge amount of sprites for me.

Am I missing something? What's the best solution here?