Rigidbody uncontrollable bouncing on uneven terrain

Hi there, I am currently working on a character controller that uses the rigidbody component. My current issue is that in some places where my terrain is a bit bumpy, the character starts bouncing like there is no tomorrow. I have been searching all over the place for a solution with not luck.

Relevant info: I am using a sphere collider. Here is a snippet of the code that I am using to move the character.

 var horizontalInput = Input.GetAxis("Horizontal");
 var verticalInput = Input.GetAxis("Vertical");
 
 Vector3 moveDirection = new Vector3(horizontalInput * Slowest, 0, verticalInput * movementSpeed);
 
 _rb.AddRelativeForce(moveDirection);

I have a video that will show the exact problem that I am having. https://youtu.be/PV54DA4PQWw

Any toughs on how to address this issue will be appreciated, thank you :)