NavMesh issue with spawning players

We use a script that says "take this objects transform, then randomly choose a position at 360 degrees and 100 units out, and spawn an enemy somewhere on the perimeter of that circle."

If I turn off the NavMesh and just run it, the script itself performs great. It will drop enemies randomly around that perimeter.

However, when I turn NavMesh on, they still spawn where they are supposed to, but some of them will "shoot" off to some other point of the map. Its almost like the NavMesh forces them off to some other "more suitable" point of the map. If I bake a mesh with 0 objects in the scene, and it is a perfectly solid blue plane, then they act fine. If I have a lot of objects in the scene and bake, then you get all the intricate geometry of the NavMesh. Which, seems to be the issue. The more complex, the more "dead" areas I have. Where now all the enemies actually span in the circle, but are immediately and very quickly shot over to some other point of the mesh that usually isn't inundated with these lines in the mesh.

To clarify, I know how to setup NavMesh parameters, and have tried an immense amount of variations. My actual ground in 100% flat through the whole thing. The only things I have really are height, to say what enemies can fit under, and radius, being how far around objects to avoid.