Shadow Z-fighting In Scene View

I'm having an annoying issue in Scene View. I have a 512x512 unit terrain with Dual Lightmaps in Deferred Rendering. I have dynamic shadows turned on. In Scene View I'm getting a lot of z-fighting near the intersections of objects, which I can live with. However I'm getting a very weird black flickering effect on everything in the scene as well, which doesn't show up in Game View but is pretty annoying. Here's a picture:

It goes away when I turn off shadows or when I switch back to Forward Rendering (I need Deferred because it plays nicely with dual lightmaps, light probes, and dynamic shadows, not really for the number of lights you can use). Playing with the shadow bias gets it to kind of go away but it doesn't go away completely. Same with changing the shadow texture size. Quality is set to Fantastic. I'm building for Windows. DX11 is enabled. No graphics emulation. Terrain and objects are using a custom shader that looks up a fog of war render texture, but the problem happens on objects with the default diffuse shader too, so I don't think that's the problem.

edit: I'm running Windows 7 64bit, I have a GTX 760 with the latest drivers.

If anyone could help me fix this it would be great.