- Home /
Stencil Shaders with depth testing
Hi, I'm building a unity level that takes advantage of Stencil Buffer shaders to mask objects. I have a shader that is puts a value in the stencil buffer, and another shader that only renders when that value is in the stencil buffer. This part is all working.
My problems arise when using multiple masks with differing values. If I have a mask that sets the value to 1 (Mask 1) and a mask that sets the value to 2 (Mask 2), and Mask 2 precedes Mask 1 in the Rendering Order (they use the same shader, so are in the same queue) then any areas of overlap on screen will set the Buffer to 2, even if 1 is in front. This is in part because the stencil buffer shader does not write to the z buffer (which it kinda can't so the the correct geometry can render behind it later). I'm also trying to prevent geometry that only renders when the buffer is set to #value from rendering when the mask that sets the buffer to #value is behind the object. I have two rough ideas for solutions, but lack the knowledge of shader coding to implement them / know if their stupid ideas.
Change the render order of the masks so that the furthest mask from the camera renders first, and the render order moves forward. This would make the overwriting work in my favor. I believe at the moment the render order (within the same point in the queue) is defined by the order of mesh creation? This solution would break down if the masks ever intersection since it would be a per object or per fragment operation but this case won't arise in my project.
Recycle the depth buffer. Have an early pass than runs on all my masks and all my geometry and writes to the depth buffer, then run all my Stencil buffer masks. Then (somehow?) clear the depth buffer, then render all my geometry. This would also mean objects in front of masks would prevent those areas from being effected by the mask. This sounds like it might be very inefficient, and I have no idea how to clear the depth buffer at a specific point in the queue (or at all).
Do either of these ideas sound viable? and can anyone help me out with understanding the pieces of shader code I'm missing? Or am I doing this all wrong and you've got a totally better way of doing it.
Hmm, do you want to know if an object is within mask 2 and/or mask 1, or would you like mask 2 to fully overwrite mask 1, but with depth testing?
I would like the masks to fully overwrite each other, based on depth testing. So 2 will fully overwrite 1 if 2 is closer to the camera.
I could build an additive mask with an incremental stencil shader, like the circles in the documentation I think?