The question is answered, right answer was accepted
Get the output of the main camera and pass it to a native plugin
Edit: The issue is now resolved. There were a few DirectX errors and mismatches like the texture format that you get from Unity and so fort. Just had to enable the debug flag of the DX device and see what's really going on.
I am trying to copy (in a efficient manner, not leaving the GPU, etc.) the output of the main camera to a native plugin I am working on. I'll have to do some shenanigans in there, but for now I'm trying to open a window and render the same picture in the plugin. The issue I'm having is I get only some gray-ish rectangle, and not the actual camera output. Here's my approach:
In Unity I created a camera that is child to the main camera. I added a script to that camera which makes it render to a texture and passes the texture to my native plugin (based on THE native plugin example):
IEnumerator Start()
{
myCamera = gameObject.GetComponent<Camera>();
if (myCamera != null && myCamera.pixelHeight > 0 && myCamera.pixelWidth > 0)
{
CreateTextureAndPassToPlugin();
yield return StartCoroutine("CallPluginAtEndOfFrames");
}
}
private void CreateTextureAndPassToPlugin()
{
var myRT = new RenderTexture(myCamera.pixelWidth, myCamera.pixelHeight, 24, RenderTextureFormat.ARGB32);
myRT.Create();
myCamera.targetTexture = myRT;
InitNativePlugin(myRT.GetNativeTexturePtr());
}
private IEnumerator CallPluginAtEndOfFrames()
{
while (true)
{
yield return new WaitForEndOfFrame();
GL.IssuePluginEvent(GetRenderEventFunc(), 1);
}
}
Then, in my native plugin I get the Unity device and context and map the content of the texture I get from Unity to be able to render it on my window:
IUnityGraphicsD3D11* d3d11 = s_UnityInterfaces->Get<IUnityGraphicsD3D11>();
ID3D11Device* unityDevice = d3d11->GetDevice();
D3D11_TEXTURE2D_DESC uDesc;
unityTex->GetDesc(&uDesc);
ComPtr<ID3D11DeviceContext> unityContext;
unityDevice->GetImmediateContext(&unityContext);
D3D11_TEXTURE2D_DESC copyTexDesc;
copyTexDesc.Width = uDesc.Width;
copyTexDesc.Height = uDesc.Height;
copyTexDesc.MipLevels = 1;
copyTexDesc.ArraySize = 1;
copyTexDesc.Format = DXGI_FORMAT_B8G8R8A8_UNORM;
copyTexDesc.SampleDesc.Count = 1;
copyTexDesc.SampleDesc.Quality = 0;
copyTexDesc.Usage = D3D11_USAGE_STAGING;
copyTexDesc.BindFlags = 0;
copyTexDesc.CPUAccessFlags = D3D11_CPU_ACCESS_READ;
copyTexDesc.MiscFlags = 0;
ComPtr<ID3D11Texture2D> txResult;
hr = unityDevice->CreateTexture2D(©TexDesc, nullptr, &txResult);
D3D11_BOX box;
box.left = 0;
box.right = uDesc.Width;
box.top = 0;
box.bottom = uDesc.Height;
box.front = 0;
box.back = 1;
unityContext->CopySubresourceRegion(txResult, 0, 0, 0, 0, unityTex, 0, &box);
D3D11_MAPPED_SUBRESOURCE unityMapped;
hr = unityContext->Map(txResult, 0, D3D11_MAP_READ, 0, &unityMapped);
For some reason I end up getting only gray from the pixels in unityMapped.pData (see below).
Do you think my approach is correct? Any ideas? Thanks!
Answer by krasi_intugame · Jun 16, 2016 at 08:43 PM
Yes. It's working now. There were a few DirectX errors and mismatches like the texture format that you get from Unity and so fort. Just had to enable the debug flag of the DX device and see what's really going on.
Hi, I try to make the exact same thing as you do, the exact same way... And I get the same problem :) Can you give a bit more details on how you resolve it ? In my case, the copy goes well in DirectX (I put a check on each call on the returned value, but the data inside my mappedData is always 0). Thank you !
@krasi_intugame: you should at least post your final working D3D11_TEXTURE2D_DESC to improve this answer.
Simply change
copyTexDesc.Format = DXGI_FOR$$anonymous$$AT_B8G8R8A8_UNOR$$anonymous$$;
with
copyTexDesc.Format = uDesc.Format;
and you will get the correct texture description according to the format you declared in the behaviour with :
var myRT = new RenderTexture(myCamera.pixelWidth, myCamera.pixelHeight, 24, RenderTextureFormat.ARGB32);
In the example above the RenderTexture is declared as ARGB32 while the copyTextDesc is declared as BGRA32.