Using GetPixel via Raycasting Without Colliders

Title probably sounds counterintuitive, but basically I have a complex character comprised of a skinned mesh with the usual diffuse/specular map and a normal map, it also has another RGBA map which does not render. This one is used purely so that collision behaviour can vary wildly across different areas of the mesh, and in an extremely precise manner.

What I have previously tried is to use physics collisions to determine the texel data at the hit point. Unfortunately this only works in certain circumstances which aren't very useful to me. For example, because GetPixel normally requires a mesh collider it cannot be used for fast moving collisions or for skinned meshes (because the mesh collider will not deform with the visible mesh).

What I therefore need is to be able to get RGBA data from a specific texel on a skinned mesh, without use of a mesh collider. Any ideas?