Player object accelerates??? C# Unity2D

Hi I've been trying to learn programming but I get stuck frequently, finally decided to try a forum.

I'm practicing by putting together a basic 2d platformer. I've been struggling all day with getting the player double jumping correctly. But now that the double jumping is in place and I've been moving around a little more in game I noticed a pretty intense acceleration which doesn't drop. The jumps also grow higher even if I'm standing still and jumping in place.

 void Update () {
     CharacterController controller = GetComponent<CharacterController>();

     if (controller.isGrounded) {

         moveDirection = new Vector3(Input.GetAxis ("Horizontal"), 0, Input.GetAxis ("Vertical"));
         moveDirection = transform.TransformDirection(moveDirection);

         moveDirection *= jumpSpeed;
         jumps = 0;
         jumpCount = 2;

         }

     if(Input.GetKeyDown (KeyCode.Space) && jumps < jumpCount) {
         Jump ();
         jumpCount = jumpCount - 1;
     } 
 

     moveDirection.y -= gravity * Time.deltaTime;

     controller.Move (moveDirection * Time.deltaTime);
      



 }

 void Jump() {
     moveDirection.y = jumpSpeed;
     jumpSpeed++;
 }