Using raycast to sonify point cloud data and geometry of shapes

Hello community,

I am working with the blind and the visually impaired to assist with object identification.

I am looking for development advice for the best approach to sonify shapes with accuracy using sound in a 3D unity scene at runtime.

I have been using existing datasets of simple objects and also iPad Pro and have seen the geometry mesh and point cloud information that can be captured. I am really interested in the communities opinion of how sonification of this data could be approached. I have a procedural and spatial audio background but am struggling with getting the dataset/data to a usable format for audio implementation.

I have looked into volumetric audio, however surfaces don’t have great definition with this method and if blindfolded you could not detect that object shape or detail of edges.

I feel as though a raycast, focused, almost an audio scrubbing method, from FPS view then spatialised would possibly be the best approach, although I want to know your thoughts, I am unsure dev wise how to implement this. Would be great to scan through the point cloud or geometry data and extrapolate it so that I could use those parameters to drive sound. I have attempted to draw this https://share.icloud.com/photos/0o81uJF3vmaz6YM5vOm-efzzQ

I would need to leverage the raw point cloud data or geometry to sonify, I am just really unsure on this and would appreciate any advice on this.

This video is unique but is actually quite interesting in terms of approach, if a user/player had control of this it would be an interesting way for a blind person to inspect a complicated object. https://vimeo.com/195054847

I would want to start with boxes and spheres first and the consideration of someone moving round the object needs consideration also.

Thanks in advance

R