Debug build runs better than release build.

When running the debug build with the Profiler we're getting a good frame rate that bounces between 55 and 60. When we make a release build the frame rate randomly drops from down to 30 for a few seconds then bolsters up to 60+ then event out again. We're not sure what is causing this. We don't see any issues in the output logs and everything runs fine in the inspector. If anyone can give any suggestions that would be appreciated.

*edit forgot to mention running Unity 5.3.4f1 on Windows 10