Problem turning character on a sloped surface.

I'm working on a custom (non-rigidbody) character controller that is capable of rotating to move across any surface, including walls and ceilings, and interacting with terrain that uses a concave mesh collider. Unfortunately, I'm having some trouble getting the character to turn properly while standing on a slope.

In order to work properly the controller must be rotated twice in each step. The first rotation occurs when the player moves, it must gradually turn the character around his local Y-axis until he is facing the direction of the input relative to the camera direction. The character is then moved by speed * transform.forward and resolves any collisions that may occur. The second rotation coincides with ground detection, the character should be rotated so that his transform.up equals the normal of the surface he's currently standing on.

This is the closest I've come to code that does what I want but it has an unfortunate tendency to flip the character upside down:

 //Turn
      if (moveInput != vector3,zero)
     {
         Vector3 targetDirection = (moveInput.z * cam.transform.forward) + (moveInput.x * cam.transform.right);
         targetDirection.y = 0;
         targetDirection.Normalize();
 
         turnDirection = Vector3.Slerp(turnDirection, targetDirection, Time.fixedDeltaTime * player.trn);
         transform.rotation = Quaternion.FromToRotation(Vector3.forward, turnDirection) * Quaternion.FromToRotation(Vector3.up, transform.up);
 
     //Move by speed * transform.forward.
     }
 
      //Resolve collisions
 
      //Ground detection
         Vector3 origin = transform.position + (20 * transform.up);
 
         if (Physics.Linecast (origin, origin - (transform.up * 36), out contact) && Vector3.Angle (contact.normal, transform.up) <= 45)
         {
             transform.position = contact.point;
             ground = true;
             transform.rotation = Quaternion.FromToRotation(Vector3.forward, turnDirection) * Quaternion.FromToRotation(Vector3.up, contact.normal);
             Debug.DrawLine(origin, origin - (36 * transform.up), Color.green);
         }
     }

The flipping seems to occur most frequently when the character is on certain parts of the collider, or when there is no collision with the ground at all, but I haven't been able to replicate the issue consistently.