3D heatmap visualization on geometry

Hi all,

I'm trying to keep track of what the main camera has been pointing at using raycasts. I want a visualization similar to that of a heatmap but in 3D: cold colors signify that the camera has only briefly pointed at the point on geometry, hot colors means that the camera has lingered at the location for longer.

I am currently storing RaycastHits that I wish to use for this visualization.

I was wondering if a shader could do the job, maybe in conjunction with the Replacement Shader technique or if there is another way of doing it? Another way could be to create a custom lightmap from spotlights placed at the orientations and positions of the recorded raycasts. Another idea I had was to somehow keep a set of vertex colors and use a custom shader to read those.

Your help is greatly appreciated.