Networking 5.1 drops framerate

I don't know if anyone else had this problem but in the new networking system it drops the framerate. Even with nothing in the scene. When I build the game.Start the server then the client.It loads up the second scene. Then drops the framerate by at least 10 to 20 when ever I click on the client window.I removed everything from the scene so Im sure it not my code doing it.Even with nothing running the framerate is still dropped.Is this a bug or am I doing something wrong.All I have is the network manager in the scene.It has a second scene with a floor and the third person character model that comes with unity.I have remove all scripts on it except the network identity.