Android Pause conflicting with action

I made a pause button for my simple game which is in the top right corner nothing special. The problem is that my controls on android are if you click on right side of screen you move right and the opposite.

     if (Input.touchCount > 0) {
         Touch touch = Input.GetTouch (0);
         float middle = Screen.width / 2;

         if (touch.position.x < middle && touch.phase == TouchPhase.Began) {
             //moveLeft;
             player.transform.position = new Vector3 (player.position.x-3, player.position.y, player.position.z);
         }

         if (touch.position.x > middle && touch.phase == TouchPhase.Began) {
             //moveRight;
             player.transform.position = new Vector3 (player.position.x+3, player.position.y, player.position.z);
         }

     }

So whenever I click the pause button it pauses but it also moves the player to the right. How do I stop that from happening. (I know I could make left and right separate buttons but I really like the way the controls are at the moment and do not want to change them).