Need help with flight script

So I'm currently working on a space flight game using keyboard and mouse. The mouse controls where you want to go by using the current mouse position and lerping towards that. So mostly it's working fine but when you start traveling straight upwards or straight down the camera and the player ship begins rotating like crazy.

Here is my code for flying,

 Vector3 halfScreenSize = new Vector3(Screen.width / 2, Screen.height / 2, 100f);       
 Vector3 lookPos = halfScreenSize + DeadZone();
 
 //Look At the Mouse
 LookAtPos = Camera.main.ScreenToWorldPoint(lookPos);
 SmoothedLookAtPos = Vector3.SmoothDamp(transform.position, LookAtPos, ref velocity, 2f, 0.1f);   
 transform.LookAt(SmoothedLookAtPos);

Any advice would be useful, also the Deadzone() function is used to create a deadzone in the middle of the screen that the mouse ignores.