How can I make my player walk like a ghost

I could make my player float above the ground and move in the direction the camera is facing, but when I collide against another object, the player rotates a little. How can I make it to just hit the wall without rotating? Secondly, I want my player to float up and down constantly like a floating ghost and not stop after not moving for a while.

This is the code I have so far:

 public float speed = 90f;
 public float turnSpeed = 5f;
 public float hoverForce = 65f;
 public float hoverHeight = 5f;

 private Rigidbody playerRigidbody;

 void Awake() {
     playerRigidbody = GetComponent<Rigidbody>();
 }

 void FixedUpdate () {
     Ray ray = new Ray(transform.position, -transform.up);
     RaycastHit hit;

     if (Physics.Raycast(ray, out hit, hoverHeight)) 
     {
         float proportionalHeight = (hoverHeight - hit.distance) / hoverHeight;
         Vector3 appliedHoverForce = Vector3.up * proportionalHeight * hoverForce;
         playerRigidbody.AddForce(appliedHoverForce, ForceMode.Acceleration);
     }    

     Vector3 targetDirection = Camera.main.transform.forward;
     if (Input.GetAxis("Vertical") != 0) {
         playerRigidbody.AddRelativeForce(targetDirection * speed);
     }
     Vector3 rotateDirection = Quaternion.Euler(0, 90, 0) * targetDirection;
     if (Input.GetAxis("Horizontal") != 0) {
         playerRigidbody.AddForce(Input.GetAxis("Horizontal") * rotateDirection  * speed);
     }
 }