koobas.hobune.stream
glReadPixels in Unity 5 always get black frame
I was following Kibsgaard's suggestion in this link text thread using glReadPixels under Unity 4 I can get very high performance video capturing feature in our game. The general idea is when game is running in openGL mode using glReadPixels in PostRender or EndOfFrame, the return buffer will contain the colors of screen pixels.
But recently after we upgrade project to Unity 5, we found this was no longer working, glReadPixels always returns black.
I tried add -force-opengl,let unity running in legacy mode. Tried bindFrameBuffer, but all these were not working for me.
Anyone knows how to fix this in Unity 5? We really need very high performance video capturing feature. I tried renderTexture solution under unity 4 which was 3 times slower than glReadPixels solution.
What I'm seeing is that it works in the editor, but fails in the player. Have you had any progress on that issue since posting this question?
Yes we have the Unity support helped us, it's solved by moving the native GL calls(glReadPixels/glGetTexImage) to the GL.IssuePluginEvent, and it works fine.
Hi kdu, great that you got it working! I have some issues with the transferred texture being from a different render texture, than the one bound by RenderTexture.active in Unity and it doesn't help to use a command buffer to bind the texture and issue the plugin. Did you get it to work reliably? If so, are you able to share the source for your plugin? Do you set the rendertexture in your plugin or from Unity? Is it with OpenGL Core or Legacy?
Hi $$anonymous$$ibsgaard, have you find the way how to use GL.IssuePluginEvent for glReadPixels/glGetTexImage? How is it posssible to send an parameterst to the glReadPixels in this native plugin call? I would be very appreciate for any advise
Hi irbis, yes I did eventually solve the problem, but it does require a bit of extra code in the native plugin. What I do now is:
Create a RenderTexture in Unity and set it as the camera's render target
Get native pointer from that RenderTexture using GetNativeTexturePtr(). Which is actually a pointer to the ColorAttachment and not the framebuffer (RenderTexture) itself.
Create a framebuffer (in the native code plugin) that uses the same ColorAttachment and store it for later use
Issues the transfer once done rendering using GL.IssuePluginEvent()
The transfer should start by binding the framebuffer, binding the colorattachment, glreadpixels and finish by unbinding the framebuffer (Unity 5+ actually cares if you clean properly up now or not)
If you don't use a RenderTexture for your camera, maybe you can bind the front/backbuffers directly ins$$anonymous$$d, however, I didn't need that and didn't find a quick way of getting a reference to those.
I just found out that Unity 2018.1 has a new function to async read back data from the GPU. $$anonymous$$aybe this is useful for you. In our case, the read back is highest priority and waiting even a couple of frames is bad.
We also have this issue on iOS and by calling glGetError() we get the message, GL_INVALID_FRAMEBUFFER_OPERATION.
It was so strange that before calling glReadPixels, we have check the frame buffer status and it's GL_FRAMEBUFFER_COMPLETE.
GLenum status = glCheckFramebufferStatus(GL_FRAMEBUFFER);
NSLog(@"Bind framebuffer : %x", status);
if (status != GL_FRAMEBUFFER_COMPLETE)
NSLog(@"Bind framebuffer error! : %x", status);
Unity version : 5.4.1f1 (649f48bbbf0f)
iOS : 9.3.5
OpenGL ES : 3.0
Anyone knows how to fix it? or give us some tips to check this?
Thanks!
By changing to $$anonymous$$etal and using CVPixelBuffer retrieve the pixel data, fix it!
I've got what you mean @kdu_timeplay . Native glReadPixels function must be put in the callback of GL.IssuePluginEvent to ensure that it is called in the render thread. While in Unity 5.1 version it is not compulsory
