Raycast is randomly going to the wrong place in VR.

Hullo, I've been having the strangest problem with my VR game! I have a gun on my players hand and it makes a line renderer to where it is pointing. Sometimes though, at from what I can tell is entirely random, the ray goes the wrong direction (every degree it is above the horizon becomes below the horizon) and attach to the opposite side of the object (the side facing away rather than toward me). If the ray is below the horizon it just straight up won't hit anything.

Here is the code (none of the variables here are modified elsewhere, and the Debug.Draw Ray uses the exact same input as the raycast, and the draw ray is in the intended direction). The hit.point has the faulty value, not just the line's end.

 void StartGrapple()
     {
         RaycastHit hit; 
         if (Physics.Raycast(gun.position, gun.forward, out hit, maxDistance, GrappleAble)) // (grappleable is a layermask)
         {//If I hit something:
             grappling = true; //start grappling
             grapplePoint = hit.point; //Store where I was pointing
             distanceFromPoint = Vector3.Distance(transform.position, grapplePoint); //find out how far from the point my hand is
             lr.positionCount = 2; //Make the cable visible
             lr.SetPosition(1, grapplePoint); //Make the cable's end be the point I hit
         }
     }