How to restrict Direction?

I'm trying to make my player climbing a wall and as far as I have been doing it's fine but the problem is that I am basing my movement of moving while climbing in my camera(TPS). The real problem with all is that the direction of the player doesn't restrict based on the wall, even if I set an specific position to stay(You will notice in the script).

 if(climbing == true){
 moveDirection = new Vector3(Input.GetAxis("Horizontal"),0,Input.GetAxis("Vertical"));
 moveDirection = camera1.TransformDirection(moveDirection);
 moveDirection *= crouchSpeed;
 transform.position = hit.point;
 angle = Vector3.Angle(hit.point-hit.transform.position, hit.transform.forward);
 if(angle > 0 && angle < 90){
 transform.rotation = Quaternion.LookRotation(-hit.transform.forward);
 transform.position += hit.transform.forward*((hit.transform.lossyScale.z/2));
 }else{
 transform.rotation = Quaternion.LookRotation(hit.transform.forward);
 transform.position += hit.transform.forward*((-hit.transform.lossyScale.z/2));
 }
 transform.rotation.x = 0;
 transform.rotation.z = 0;
 transform.position.y = hit.transform.position.y+(hit.transform.lossyScale.y/2)-0.7;
 }

So the question would be how do I restict the direction of the player to only move based on the wall(right, left) considering the camera