How to properly move a rigidbody character?

So I've written a rigidbody character movement script which sets the velocity directly... the script is kinda long so I'll just post the crucial part of it

 Vector2 input = GetInput();        
 if (Mathf.Abs(input.x) > float.Epsilon || Mathf.Abs(input.y) > float.Epsilon)
 {
     Vector3 direction = GetDirection();
     transform.localRotation = Quaternion.LookRotation(direction);
 
     UpdateTargetSpeed(input); // update target speed based on movement direction
     Vector3 movementVector = new Vector3(input.x * TargetSpeed, 0f, input.y * TargetSpeed); // add speed to input vector
     movementVector = transform.localRotation * movementVector; // rotate the vector to where the character is facing
     movementVector = Vector3.ClampMagnitude(movementVector, TargetSpeed); // clamp magnitude to avoid faster speed on diagonals            
 
     body.velocity = new Vector3(movementVector.x, Velocity.y, movementVector.z);
 }

This works fine but I keep reading that setting velocity directly is a bad practice and that I should use addForce with ForceMode.VelocityChange

I've tried doing that but the movement is not smooth as input gets ignored when desired velocity is reached and then the body slows down due to drag and/or friction and then the force is applied again, etc...

this is a basic example (without rotation) of what I've tried

 Vector2 input = GetInput();
 
 if (Mathf.Abs(input.x) > float.Epsilon || Mathf.Abs(input.y) > float.Epsilon)
 {
     Vector3 force = new Vector3(input.x * 5f, 0f, input.y * 5f);
             
     if (body.velocity.sqrMagnitude < speed * speed)
     {
         body.AddForce(force, ForceMode.VelocityChange);
     }
 }

Is there any way I can get the same behavior I got from my direct velocity setting example but using AddForce or something else that is more "correct" than setting velocity correctly?

Also why exactly is setting velocity a "bad practice"? I haven't been able to find a thorough explanation for that so if anyone can share some knowledge I'd appreciate it.