How to add interactivity (move/raycast objects) to a mostly GPU-based particle system

Hi everyone,

this is my first post and I am still on the left side of the learning curve, so please be gentle with me :). A few month ago (my first modern Gameengine dev experience) I started tinkering on a particle(-alike) system, in which by now most work is done on the GPU side. A compute shader is doing the calculations which positions the objects, a geometry shader is drawing the not so important object like lines, etc. The major objects are traditional GameObjects, quite a few of them (>5k), but only around 20 different prefabs, being instanciated as needed. These are also moved based on the calculations of the computeshader. Ideally these GameObjects would live for performance reasons on the GPU side as well. What kept me from moving them over so far has been that these objects need to be selectable. Being normal GameObjects, they also offer the option to make them look good easily (contrary to procedurally created meshes/primitives).

Would it be possible to have these objects selectable, even when living only on the GPU? This is where I am still searching for a good idea. If anyone has a suggestion on how to do this, I would greatly appreciate your idea.

What I so far had been thinking of is:

• Keeping them as normal GameObjects (instanciated, present in the scene editor) and update their position with in the vertex shader on basis of a computebuffer. This way they would still have normal functionality like raycasting, and it would reduce the load of reading back the computebuffer every frame. Would it be possible to move an object inside a shader to a different world position?

• Have them generated procedurally with the geometry shader (as primitives, loosing the (easy blender) good looks). But here again I am missing a simple idea of how to make same selectable. Back in the old days I remember that storing the object index alongside the pixel coordinates used to be common practice, but this sounds a little deprecated nowadays :).

Looking forward to hearing your input and ideas.

Bammi