SmoothDamp function affecting controller.isGrounded

So i have an odd problem that seems to affect the true or false output of isGrounded on my character controller.

So first I have this bit of code in my Update method that checks if the controller is grounded and if it isn't, it applies gravity:

  void Update()
     {
 
         Debug.Log(controller.isGrounded ? "GROUNDED" : "NOT GROUNDED");
 
         // Checks if player is grounded, if not, apply gravity.
         if (controller.isGrounded)
         {
             moveVector.y -= emptyNum;
         }
         else
         {
             moveVector += Physics.gravity * Time.deltaTime;  
         }
 
         if (canMove == true) Move();

then in my Move() function i have this bit of code:

 void Move()
     {
         Vector3 direction = new Vector3(Input.GetAxisRaw("Horizontal"), 0, Input.GetAxisRaw("Vertical"));
         Vector3 targetMovement = direction;
 
         if (direction.magnitude > 1) direction.Normalize(); // keeps the Vector's magnitude at 1
 
         if (direction != Vector3.zero) qTo = Quaternion.LookRotation(direction); // Set's the look rotation based on our direction
 
         transform.rotation = Quaternion.Slerp(transform.rotation, qTo, Time.deltaTime * moveSpeed);
 
          moveVector = Vector3.SmoothDamp(moveVector, targetMovement * moveSpeed, ref currentMoveVel, moveSpeedSmooth);
 
         controller.Move(moveVector * Time.deltaTime);
     }

So i figured out if I remove the SmoothDamp() function then isGrounded debugs correctly (it'll say "GROUNDED" when I am grounded and "NOT GROUNDED" when i'm not). HOWEVER, without SmoothDamp() i cannot move.

I found that the reason SmoothDamp is giving me issues is that I would use moveSpeedSmooth variable in place of Time.deltaTime in order to not slide around on the ground, however when I do this Unity outputs GROUNDED and NOTGROUNDED as true at the same time.

So basically, I can fix the isGrounded issue by using Time.deltaTime, however that causes my player to slide around on the ground which is game-breaking.

So how can I have it both ways? Thanks in advance for the help.