Creating a 3D portal effect using stencil buffer with a deferred shading setup?
I'm looking to implement the effect shown here. I got it working fine in the scene view but too late I realized that the implementation described here simply doesn't work if the camera is set to use deferred shading.
There's some discussion online about why the stencil buffer doesn't work with deferred shading, but I've yet to find a clear alternative.
I need a flat 'screen' which appears to have a 3D object inside it, as in the example shown. I also need to have any number of these screen/object pairs such that if you place two pairs next to each other, each object occupies its own 'space': you wouldn't be able to see Object A through Screen B, or Object B through Screen A.
Has anyone worked on this and knows a way to get around this limitation? Thanks.
Answer by b4th · Mar 22, 2019 at 10:16 AM
You're correct, due to known issues with stencilling in the Deferred rendering pipeline, stencil shaders will only work with Forward rendering.
So your best bet, and actually the more common approach to portal rendering, is to use a Render Texture. If you place a secondary camera at another point in the world, have it move and rotate in exactly the same way as your Main Camera, and have it render to a texture with exactly the same dimensions as the screen, you can then use a custom shader to map that render texture to your portal object in screen space (rather than mesh UV space).
Depending on your needs, you could even place your secondary camera at the same position in the world as your Main Camera, even make it a child in the hierarchy, and change what each camera can see using their Layer Culling Masks, culling layers from the Main Camera that you don't want to see outside the portal, and culling layers from the secondary camera that you don't want to see inside the portal.
The same principle is commonly used to create mirror reflections (e.g. for flat water), but you would need to mirror the Main Camera's position and orientation around the reflection plane.