Serial Communication Running Slow

I have connected IMU (Gyroscope&Acc&Magnetometer) to my Unity3D project and using serial communication via USB.

The problem is, when my script becomes little heavy with gaming codes, serial communication slows down - when values from IMU are supposed to change at certain point of time, they change after couple of seconds and as time goes, data stream cannot catch up with the game.

I am calling myPort.ReadLine() from Update function to read serial data from COM port.

What is the solution? - If I understand the problem right, I want the serial data reading not to be waiting to my app's next frame to receive new values.

May reducing Baud Rate of the IMU device work?