UWP - Getting access to an IDirect3DSurface in Unity
Hello,
I am working on a UWP project, where I grab a camera's video frames using a MediaCapture. I can get the SoftwareBitmap or the IDirect3DSurface (from the VideoMediaFrame), and although things work with the SoftwareBitmap (CPU) approach, I need better performance, and would much rather prefer to stick to the GPU (IDirect3DSurface) path.
My problem is that, although I can get the IDirect3DSurface from the MediaCapture, I don't understand how I can get access to this image to work with it. For example, I would like to check the image by creating a Texture2D and applying in a UI element so I can see what I am receiving from the MediaCapture. With the SoftwareBitmap I can get the byte buffer as described on the Microsoft website, and create a Texture2D ifrom that, but it is slow of course. I have seen that there is a function to "Create" a Texture2D from an external pointer, which would be great, but I don't get if/how I can get this pointer directly from the IDirect3DSurface. Besides wanting to show the image as a Texture2D ideally without copying it (especially on the CPU), I would like to run some shaders over the image. If I can create a Texture2D out of it, it should be fairly easy to run shaders and everything over it, though.
If anyone has any ideas of how to bridge this gap, it would be great to hear about it! :)