8 directional movement rotation problem

My character moves in 8 directions and rotates to the direction he is walking towards. The problem however is when it's walking diagonally and I release the keys to stop the character. What happens is that when I release the directional keys, my character would face towards the key I released last. Unless I release the directional keys on the exact same frame, the rotation keeps turning away from the diagonals.

Example: Pressing W & A -> Character moves and turns towards northwest -> Release W & A together but not on exact same frame -> Character stops moving and turns to either north or west depending on which key I released slower (W / A).

 public class PlayerController : MonoBehaviour {
 
     private CharacterController cc;
     Vector3 direction;
     float speed = 14f;
 
     void Start () {
         cc = GetComponent<CharacterController>();
     }
     
     void Update () {
         direction = new Vector3(Input.GetAxisRaw("Horizontal"), 0, Input.GetAxisRaw("Vertical"));
 
         if (direction != Vector3.zero)
         {
             transform.rotation = Quaternion.LookRotation(direction);
         }
 
         cc.Move(direction * speed * Time.deltaTime);
     }
 }