Problems implementing TPP camera that doesn't go through walls

I am trying to develop third person camera that doesn't go through walls and so far I'm failing at it miserably ;(.

First I've got code from some site with Unity snippets and tutorials (don't remember URL, although if you'll google code, you should find it). Of course, as most solutions that are available for free on the web, this camera passed through walls. My friend suggested that I cast ray from player in direction of camera with length roughly same as distance between the two, and if collide with something, put camera on this point instead of maximum one. But unfortunately code that puts camera at point of ray collided with object is somehow borked, as camera upon collision with wall gets something that can only be called epileptic surges.

Here's the code:

 using UnityEngine;
 public class FollowCam : MonoBehaviour {
     public GameObject target;
     public float rotateSpeed = 5;
     Vector3 offset;
     float basedistance;
     public float mindist;
     void Start() {
         offset = target.transform.position - transform.position;
         basedistance = Vector3.Distance(target.transform.position, transform.position);
     }
     
     void LateUpdate() {
         float horizontal = Input.GetAxis("Mouse X") * rotateSpeed;
         target.transform.Rotate(0, horizontal, 0);
         Ray ray = new Ray();
         ray.origin=target.transform.position;
         ray.direction = this.transform.position;
         RaycastHit rayhit = new RaycastHit();
         if (!(Physics.Raycast(ray,out rayhit,basedistance*2))){
             float desiredAngle = target.transform.eulerAngles.y;
             Quaternion rotation = Quaternion.Euler(0, desiredAngle, 0);
             transform.position = target.transform.position - (rotation * offset);} 
         else {
             float desiredAngle = target.transform.eulerAngles.y;
             Quaternion rotation = Quaternion.Euler(0, desiredAngle, 0);
             transform.position = target.transform.position - (rotation * rayhit.point);
         }
         
         transform.LookAt(target.transform);
     }
 }

and here's a gif of what happens after camera collides with wall: https://dl.dropboxusercontent.com/u/210143/padaka.gif (no, after making ray collide with a wall, I didn't move mouse any further).

As for basedistance*2, it is that way, because I've seen that camera passes through wall a bit before going into epileptic surge which meant ray didn't hit even though it should. That seemed to fix that issue, leaving epilepsy of camera.

Could someone help me? I've tried to fix this code however I could and nothing I did helped.