Fixing CharacterMotor

So I'm using the standard (but a heavily modified) CharacterMotor for my game. There's one issue I've come across which happens with sliding.

The problem is, if you are sliding down a hill, but facing up the hill and do a little jump while holding forward, when you land you just slide up the hill like you're on the flat.

After a lot of debugging I've found that this is because the ground normal isn't updated as the following line returns false in this specific case:

private void OnControllerColliderHit (ControllerColliderHit hit)
{
    if (hit.normal.y > 0 && hit.normal.y > _groundNormal.y && hit.moveDirection.y < 0)
    ...
}

Because

    hit.moveDirection.y == 0

As the collision is registering in the sides. I've tested this on 30, 45, 60, 75 degree slopes and they all share the same problem.

I believe that specific condition is there to make sure you are colliding with the ground not the roof or walls. So obviously doing this,

    if (hit.normal.y > 0 && hit.normal.y > _groundNormal.y && hit.moveDirection.y <= 0)

While fixing the specific issue causes other issues.

I've tried fixing it but I'm struggling to think of a solid solution. Not really sure how I should detect when I'm running into a wall compared to running into a cliff.

Any suggestions?

-edit-

Oh and before anyone suggests it's due to my modified script, I've already tested with the vanilla script and the issue still exists.

I'll make a video if my description of the issue is a bit vague.