High Cpu consumption on Load Scene Async

I am having scene loading issues in Unity 2018.4 (LTS) version. My editor freezes when a scene loads. Sometimes it takes 4 sec and the other time it takes 40 sec to load same scenes. What I have noticed that while a scene is loading the profiler freezes and then after some time it shows large spike/spikes in the CPU usage area for the first/ rest of the frames (10000 ms sometimes) and then it comes down to 15ms (~ 60 fps) for rest of the frames. I am highly suspecting that when I load a scene using LoadSceneAsync somehow it results into high CPU Usage. I also tried turning on/off V-Synch and changing time-slice, ring-buffer values but nothing helped. Here is the screenshot of profiler analysis during scene loading : - alt text Please help me!