- Home /
Get object from scene (world space) from a grayscale saliency image (screen space)
I'm developing a salience tool inside Unity3D. My aim is to highlight an object model in the scene when it is the most salient in the player's view, not real-time.
What I have in mind now it to move the camera around in the scene and have it take screenshots on a interval. After this 'recording' is done. I run an external program that generates a salience map of the screenshots (a grayscale texture where white means very attention grabbing).
Now I want to replay my recording in Unity3D, but instead of taking a screenshot at each position, I place the salience image "on top of the camera's view" and want to get the object(s) that are located behind the white spot(s).
An example:
The original screenshot:
Would result into an salience image like this...
I would want to 'project' this grayscale image onto my scene from the camera's view and get the object(s) behind the most white places. Here it would be: The Waterfall, The lantern, The house by the lantern.
I heard this might be possible by use of Stencil Buffers, but I have no idea if that's true. Could somebody confirm that? Or give other solutions? All help is welcome :)
PS: just thought of a new (possible) solution, maybe i could do a raycast from every pixel in screen space that has a high value (240+) (thus white) to world space and get the hit result. Maybe this is a easier solution? I don't know much about Stencil Buffers, so maybe that's still the best route (?), but i would like to explore that subject :)
Your answer
Follow this Question
Related Questions
Possible to use sprite mesh for UI.Image GraphicRaycaster culling? 1 Answer
Finding a pixel in an images texture by raycast 0 Answers
How to get solve two or more UI images alphas? 0 Answers
Touch count & Raycasting on mobile 0 Answers
Grid Pathfinding 0 Answers