Character Controller Capsule collider making camera Jitter

Hello, I know such questions have been asked already, but none provide a solution to my problem which is more specific.

I don't have a rigidbody and no intention of using one, I have made a custom camera script and a character script that controls the character (3rd person game). The camera is controller via LateUpdate and the character via Update using charactercontroller.move

My Character only has a character controller component and the script that I made. The jitter specifically occurs when the character walks in the camera forward direction and no other occasion

It turns out that the character controller (the component ofc) is causing the jitter, if I turn it off theres no jitter at all, smooth as butter. also when I change the radius of the capsule collider on the character controller the jitter intensity changes accordingly - the thinner the collider the less the jitter.

I definetly need no jitter at all, so I ask how can I make the character controller's collider stop making my cam jitter?? I've also tried turning off the gravity, but the result is the same.

Character controllers slope and step offset comes in handy since the character needs to be able to traverse stairs, however is theres no way to stop the jitter I'd like to make my own way of stair climbing, so if you can at lest then point me towards a good tutorial or give me ideas.