Raycast pathfinder nodes

Hey

I'm doing some a* grid pathfinding and at each new node I check if I have direct line to the destination using raycasting, so I don't have to keep pathfinding if not neccessary.

this is the code:

int layerMask = 1 << 9;

 if (!Physics.Raycast(new Vector3(curPathNode.Position.x, 0.1f, curPathNode.Position.z),
 new Vector3(dest.x, 0.1f, dest.z), Vector3.Distance(curPathNode.Position, dest), layerMask))

{ //finish pathfinding, found a direct path from curPathNode to dest }

9 is the layer number I use for my wall (wall is made out of 6 planes) curPathNode is the current visited node, dest is the final destination, I rise them by 0.1f because they have a height of zero

I'm getting this (using gizmo draw lines on my path nodes and path lines): http://img687.imageshack.us/f/unledtr.png/

it does actually ignore a direct line for a while, but it goes through a wall at some point :(