CharacterController's collision rather off

I'm trying to use the CharacterController class for a simple runner game. And I found out that its collision is often wrong. Look at this screenshot, for example:

FYI, the red sphere is the point of collision reported by the OnControllerColliderHit() callback (drawn from OnDrawGizmos()). While you can see that capsule is the CharacterControl object in question. There are tons of other weird and wrong collision behaviours sometimes. Like not detecting a collision with another object when it should be, or colliding of an object althought its still far away, etc.

I didn't do anything out of the ordinary to the CharacterController's object. Just the usual calculate the final move vector of the object, time it by Time.deltaTime, use it as the parameter of CharacterController.Move()

Can anybody tell me what's wrong with this? And how to fix it?

Thanks.