[Solved] Rendering to a specific Texture2DArray slice
Hi there!
I'm currently trying to build a post-effect shader, attached to a camera, based on a number N of previously rendered frames.
To store them, I'd like to use a RenderTexture with a single Texture2DArray color render buffer. This Tex2DArray would then be used as a circular buffer: at each rendered frame, the index i in the Tex2DArray is incremented (modulo N frames), the rendered frames is copied in the ith slice of the Tex2DArray, and finally the whole Tex2DArray is processed in a subsequent shader.
My problem is that in the final shader that uses the Tex2DArray, it seems that only the slice 0 is valid, and it is always the last rendered frame (so rendering in a specific slice does not seem to work in my case, I always write in the first one).
I've read the documentation (https://docs.unity3d.com/Manual/SL-TextureArrays.html) and related answer (https://docs.unity3d.com/Manual/SL-TextureArrays.html), but neither helped me solve my problem.
Here is a simplified version of the code I use (where I scrapped obvious inits, runtime re-allocs, etc.) ; if someone could point out what I'm doing wrong, it would be great ! :)
The shader at the end is supposed to render N vertical stripes on the screen, each stripe being extracted from one specific slice of the Tex2DArray. However, in my renders only the first stripe (slice 0) is not black, and is contains always the last rendered frame.
Also, I'm pretty new to Unity / C# / D3D, so if I wrote something silly / pointless / etc. also not directly related to my problem, feel free to correct me! :)
Thanks a lot in advance for your time!
C# script attached to the camera
public class MyClass : MonoBehaviour
{
[...]
private int _samples = ... // number of frames in Tex2DArray
private RenderTexture _frames = null; // will hold the Textute2DArray
private Material _matProcess = null; // material used to process the whole _frames
void OnEnable()
{
// Testing for 2D array textures support, creation of Material with the final shader, etc.
[...]
}
[...]
void OnRenderImage( RenderTexture source, RenderTexture destination )
{
// Power of two dimensions needed for Tex2DArray
int potw = Mathf.NextPowerOfTwo( source.width );
int poth = Mathf.NextPowerOfTwo( source.height );
if ( _frames == null )
{
// RenderTexture creation without a depth buffer
_frames = new RenderTexture( potw, poth, 0, RenderTextureFormat.ARGB32 );
_frames.dimension = UnityEngine.Rendering.TextureDimension.Tex2DArray;
_frames.volumeDepth = _samples;
_frames.useMipMap = false;
_frames.autoGenerateMips = false;
_frames.filterMode = FilterMode.Point;
// Given https://docs.unity3d.com/ScriptReference/Graphics.SetRenderTarget.html
// I tested to the sRGBWrite flag, without success
// GL.sRGBWrite = false;
_frames.Create();
}
// Record last frame
if ( _lastFrameCount != Time.frameCount )
{
_lastFrameCount = Time.frameCount;
_lastFrameIdx = ( _lastFrameCount % _samples );
// Setting the render target to the proper Tex2DArray slice
Graphics.SetRenderTarget( _frames, 0, CubemapFace.Unknown, _lastFrameIdx );
// Blitting the source frame in the Tex2DArray
// Here I also tried to blit with a specific shader that reads the input texture
// with no more success (just in case)
Graphics.Blit( source, _frames );
// I also tried to use instead an explicit copy texture, but end up with an error:
// "Graphics.CopyTexture can only copy between same texture format groups
// (d3d11 base formats: src=9 dst=27)"
// Graphics.CopyTexture(
// source, 0, 0, 0, 0, source.width, source.height,
// _frames, _lastFrameIdx, 0, 0, 0
// );
}
// Finally, post-process all previous frames
// Side question: does it need to be set at each rendered frame?
_matProcess.SetInt( "_samples", _samples );
_matProcess.SetTexture( "_frames", _frames );
Graphics.Blit( source, destination, _matProcess ); // 'source' not used directly
}
}
A simple shader that uses the Tex2DArray, that actually display N vertical slices from the Tex2DArray.
Shader "Hidden/Tex2DArrayProcess"
{
Properties
{
_frames( "frames", 2DArray ) = "white" {}
_samples( "samples", Int ) = 8
}
SubShader
{
// No culling or depth
Cull Off ZWrite Off ZTest Always
Pass
{
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
#pragma target 3.5
#include "UnityCG.cginc"
//------------------------------------------------------------------------------- //
struct vertInput
{
float4 vertex : POSITION;
float2 uv : TEXCOORD0;
};
struct vertOutput
{
float4 screenPos : SV_POSITION;
float4 uvs : TEXCOORD0;
};
//------------------------------------------------------------------------------- //
UNITY_DECLARE_TEX2DARRAY( _frames );
int _samples;
//------------------------------------------------------------------------------- //
vertOutput vert ( vertInput v )
{
vertOutput o;
o.screenPos = UnityObjectToClipPos( v.vertex );
o.uvs = float4( v.uv, 0.0, 0.0 );
return o;
}
//------------------------------------------------------------------------------- //
fixed4 frag( vertOutput vInput ) : SV_Target
{
float3 uvs = vInput.uvs.xyz;
// We select the slice based on horizontal tex coord, leading to vertical stripes
// from different slices
uvs.z = floor( uvs.x * _samples );
return UNITY_SAMPLE_TEX2DARRAY( _frames, uvs );
}
ENDCG
}
}
}
For the record, I looked at the RenderTexture content with RenderDoc (awesome tool by the way!), and as I thought only the first slice is filled with the last rendered image, as if the SetRenderTarget call was not accounted for.
Hi !
Your problem is very simple :)
On each rendered frame you do
_frames = new RenderTexture( potw, poth, 0, RenderTextureFormat.ARGB32 );
where you initialize a new RenderTexture that contains your circular list of frames. So all frames are initialized to 0, and you only set tu current frame rendered. In other words, on each frame you reset your buffer.
You should initiliaze your _frames only once in the start method, and then set the rendered frame in the OnRenderImage at the correct index of your array as you do.
A good rule of thumb when dealing with images that you set to a shader is to print them in png on the CPU side (in the behaviour) to be sure that the data sent to the shader are correct.
Hope it helps ;)
Thanks for your feedback @AtGfx!
However, I guess I'm not re-creating the RenderTexture at each frame, as I test it beforehand:
private RenderTexture _frames = null;
[...]
if ( _frames == null ) {
_frames = new RenderTexture( potw, poth, 0, RenderTextureFormat.ARGB32 );
[...]
There should be no reason why the constructor is called more than once if no one destroys the reference, right? (except in the OnDisable() method).
This is consistent with the debugger that tells me I don't create it on every frame :)
Thanks anyway!
You are right, my bad, I missed this line.
By the way can you print your buffer in png to check where your problem is ? If all your images are correct in the behaviour (I mean not only the current is not black), then your problem should be in your shader. You can use
byte[] bytes = texture2D.EncodeToPNG();
File.WriteAllBytes("your file name", bytes);
Edit : you can also record your images in separate folder at each frame, and do it _samples time. You will be able to check what happend to your buffer at each rendered frame.
Answer by gsourima · Apr 26, 2017 at 01:22 PM
So... I finally managed to make this code work properly.
I followed my intuition (as stated in a comment above) that the call to Graphics.Blit(...) may not be suitable / compatible with setting a specific slice as a render target. As such, I replaced the line
Graphics.Blit( source, _frames );
with the following lines, where I simply draw a full-screen quad manually (here _matStoreFrame is a material with a simple pass-through shader that copies the source to the destination) :
GL.PushMatrix();
GL.LoadOrtho();
_matStoreFrame.SetTexture( "_CamTex", source );
_matStoreFrame.SetPass( 0 );
GL.Begin( GL.QUADS );
GL.TexCoord2( 0, 0 );
GL.Vertex3( 0, 0, 0 );
GL.TexCoord2( 1, 0 );
GL.Vertex3( 1, 0, 0 );
GL.TexCoord2( 1, 1 );
GL.Vertex3( 1, 1, 0 );
GL.TexCoord2( 0, 1 );
GL.Vertex3( 0, 1, 0 );
GL.End();
GL.PopMatrix();
I don't know if there's a simpler / more compatible way to draw this full-screen quad (besides blit :)), but now each rendered image can be stored at the proper slice in the Texture2DArray, and then read back in the final compositing shader.
Huge thanks to @AtGfx for helping me out throughout this journey! :)
Answer by Jason-Michael · Jul 05, 2019 at 08:05 AM
You could try using Graphics.CopyTexture(rt, 0, rtArray, sliceIndex);
here is my example of blur a slice in a texture2darray object,
RenderTexture rtArray = new RenderTexture(width, height, depth, format);
rtArray.dimension = TextureDimension.Tex2DArray;
rtArray.volumeDepth = sliceNum;
RenderTargetIdentifier rti = new RenderTargetIdentifier(rtArray, 0, CubemapFace.Unknown, slice);
CommandBuffer cb = new CommandBuffer();
cb.SetGlobalFloat("_DistanceScale", blurSMSampleDistance);
cb.Blit(rti, tempRT1, m_blur2DArrayMat, 0); // horizontal //
cb.Blit(tempRT1, tempRT2, m_blurMat, 1); // vertical //
Graphics.ExecuteCommandBuffer(cb);
cb.Clear();
Graphics.CopyTexture(tempRT2, 0, rtArray, slice);
Your answer
Follow this Question
Related Questions
Creating texture array contains only blank white image slices 0 Answers
Using graphics.DrawMeshNow to update a render texture 0 Answers
Problem with camera Rendering 2 Answers
How to pass a RInt RenderTexture to a ComputeShader? 0 Answers
How to draw mesh on texture,How to draw Mesh to texture 0 Answers