Buttons and Event Triggers kill FPS

I'm moving the camera like so:

 void Update () {
 
             float moveX = 0f;
             float moveY = 0f;
 
             if (moveCameraUp || Input.GetKey (KeyCode.UpArrow) || Input.GetKey (KeyCode.W)) {
                 moveY += options.cameraSensitivity;
             }
             if (moveCameraDown || Input.GetKey (KeyCode.DownArrow) || Input.GetKey (KeyCode.S)) {
                 moveY -= options.cameraSensitivity;
             }
             if (moveCameraLeft || Input.GetKey (KeyCode.LeftArrow) || Input.GetKey (KeyCode.A)) {
                 moveX -= options.cameraSensitivity;
             }
             if (moveCameraRight || Input.GetKey (KeyCode.RightArrow) || Input.GetKey (KeyCode.D)) {
                 moveX += options.cameraSensitivity;
             }
             if (moveX != 0f || moveY != 0f) {
                 Vector3 pos1 = Camera.main.transform.position;
                 pos1.x += moveX;
                 pos1.y += moveY;
                 Camera.main.transform.position = pos1;
             }
 }

And the camera has been working very well. However, I noticed that when the mouse is hovering over a button on the Canvas UI, the frames will drop down to around 20 fps. As soon as I move the mouse off the button, it continues to move around effortlessly at over 100 fps.

I've also noticed that if I keep the camera stationary, and simply keep the mouse moving over the button UI, the frames will continue to drop, usually around 40 fps, and as soon as I stop moving the mouse, or move it off the button, it returns to normal.

I'm wondering if this is normal? Is Unity suppose to have such extreme performance issues with simple buttons? or could this be another underlying issue?

EDIT: I've now noticed that it is only with buttons that have Event Trigger components attached. I'm using these to detect when the mouse is hovering on the button, and display tooltips and such. Is there perhaps another way to do this?