What is the most efficient way to sample the color at a point on a mesh?

For an arbitrary mesh, you can use Collider.Raycast() (which is more efficient that Physics.Raycast() since in casts against a specific collider). The RaycastHit structure returned contains a textureCoord, which is the UV of the hit point. You can then use Texture2D.GetPixel() or Texture2D.GetPixelBilinear() to get the pixel. If you are going to be doing the sampling a lot, it would be better to do a Texture2D.GetPixels32() to get all the pixels once, and then use the array instead of GetPixel(). The array returned by GetPixels32() is 1D, so it will take a bit of code to first convert the UV to pixel, and then figure out how to map the 2D pixel location to the position in the 1D array. Note that the texture must be marked as read/write enabled in order to read from it.