Direct3D12 native plugin render to screen

I've gathered some information for you on this from the DX12 developers:

"When you do GL.IssuePluginEvent, I think the render texture will already be set"
https://github.com/Unity-Technologies/NativeRenderingPlugin

"They can't draw "Directly" to the screen, they draw to whatever buffer the camera is set to I believe. If you look at the DX11 code, you'll see it sets up all drawing state except RTV/DSV which means those are already setup for you"

"You can also use GetNativeTexturePtr on a render texture, get the ID3D12Resource from there and have the plugin create their own RTV/DSV "

 

Hi @JDeConinck_PresenZ,

The current d3d12 native plugin interface only provides access to our internal ID3D12CommandQueue. What you really need is access to our internal ID3D12CommandList. Alternatively, if you could execute a command list of type bundle, then you could inherit the resources and state bound to the engine's internal pipeline state object and command list. This is similar to how our Metal and Vulkan native plugin implementations work. Unfortunately, our d3d12 native plugin implementation does not provide access to our internal command list and the interface's ExecuteCommandList() method does not support executing command lists of type bundle. If you try, you will get GPU validation issues. This is because internally Unity issues a direct call to its ID3D12CommandQueue::ExecuteCommandLists() which does not support bundled command lists.

Due to these constraints, there are some less performant options you could try.

You may be able to exploit the Display class to gain access to the actively bound render surface. Please take a look at the following API
https://docs.unity3d.com/ScriptReference/Display-colorBuffer.html

Display.main.colorBuffer will provide access to the active bound color render surface. Likewise, Display.main.depthBuffer will provide access to the active depth render surface.
These render surfaces are unity specific objects that contain unity specific data describing the render surface in addition to the native GPU resource backing the surface (i.e. ID3D12Resource*).

We do not have a C# side API that will allow you to extract the ID3D12Resource* from C# script. However, we do have a native rendering interface method that will allow you to do that in C++. It is defined like so
ID3D12Resource* IUnityGraphicsD3D12v5::TextureFromRenderBuffer(UnityRenderBuffer * rb);

Therefore if you feed the TextureFromRenderBuffer() method a UnityRenderBuffer, which is what Display.main.colorBuffer represents, then you can gain access to the underlying Unity owned ID3D12Resource* backing the actively bound render texture.

If you choose to use this approach, you will need to create your own d3d12 pipeline state object, command allocator, and command list. Your command list will need to bind the color render buffer via a call to OMSetRenderTarget() on your d3d12 command list.

When filling out a description for your ID3D12PipelineState object, you may struggle to define the D3D12_GRAPHICS_PIPELINE_STATE_DESC::RTVFormats and D3D12_GRAPHICS_PIPELINE_STATE_DESC:: DSVFormat DXGI_FORMATs as we don't currently provide those to you in C# or C++. Therefore my recommendation is to take a RenderDoc capture of the scene without executing your native rendering plugin. From there you can examine Unity's internal PSO and the various settings we use based on your player, graphics, quality, and camera settings.

Once you are done recording into your d3d12 command list, you will then be able to use the native plugin's interface method, ExecuteCommandList(), to execute the command list on Unity's internal d3d12 command queue.

Another approach you could try is blitting your render into Unity's actively bound render texture. For this approach, you would again need to create your own d3d12 pipeline state object, command allocator, and command list. You would also need to create your own color and depth surfaces and direct your command list to render into them. Next, you would need an API to inform your C# script driving the native rendering plugin as to what the pointer address is to your native d3d12 shader resource view pointing to your color surface. Once you have done that, you can pass the SRV pointer into a method like Texture2D.CreateExternalTexture().
https://docs.unity3d.com/ScriptReference/Texture2D.CreateExternalTexture.html

Unity will then create a Texture2D that wraps your native rendering plugin's resource without creating any internal d3d12 resources within Unity. Once you have done that, you can then invoke a Graphics.Blit() call in your camera's OnPostImage() or OnRenderTexture() methods.

Obviously blitting will incur a performance cost but I believe it may be the only viable workaround if the Display approach doesn't work for you.

I hope that helps!