koobas.hobune.stream
RawImage flicker when changing from Texture2D to VideoPlayer texture
Using Unity 2017.3
I have a RawImage into which I display a sequence of images loaded as Texture2D's. Works perfectly and seamlessly.
I then show a video into the same RawImage, using the sample VideoPlayer code, and assigning rawImage.texture = videoPlayer.texture
The video plays perfectly well, but as part of switching from the still images to the video, there is a noticeable flicker in the RawImage, as if there's a frame or two of black displayed. The first frame of the video matches the last static image I displayed, so I had expected the transition to be pretty seamless.
Note that the video has been "Prepared" prior to all this - I yield until videoPlayer.isPrepared returns true, and only then tell the video to play and set the texture.
I thought maybe there was an issue with the texture not being quite ready, but I tried yielding once or twice after calling Play and before setting the texture, but that didn't have any effect on the flicker.
I saw this item: https://answers.unity.com/questions/1294011/raw-image-flickering-when-texture-changed.html which suggests that this is something to do with material instances being set up. I don't fully understand the solution presented in that answer, nor do I understand how I could adapt it to my own case, but maybe it means something to those more skilled in Unity than I.
Any suggestions on how to get rid of that flickery frame?
Well, I came up with a solution and, I think, somewhat of an explanation.
First, VideoPlayer.frame is documented as "The frame index currently being displayed by the VideoPlayer." This is not strictly true. Or, maybe it is somewhere in the VideoPlayer's pipeline, but it's not the frame that's observable by code using the VideoPlayer's texture.
When you Prepare the VideoPlayer, at least in the mode I'm using it, the VideoPlayer creates an internal RenderTexture. You would think that, once the player has been prepared, that texture would contain the first frame of the video. It doesn't. There is a very noticeable delay before there's anything there. Thus, when my code set the RawImage texture to the player's texture, it was arranging for a texture that was, at least at that moment, empty to be displayed. This perfectly explains the black flicker, since that's the color of the background Canvas.
So my first attempt at a solution was to insert the loop here:
_videoPlayer.Play();
while(_videoPlayer.frame < 1)
{
yield return null;
}
destImage.texture = _videoPlayer.texture;
between Play and the texture set.
I figured that, despite the documentation, maybe frame was the frame about to be displayed. If so, this should result in the first (0th) frame already being in the buffer, and would get rid of the flicker. Nope. Still flickered. But when I changed to
_videoPlayer.Play();
while(_videoPlayer.frame < 2)
{
yield return null;
}
destImage.texture = _videoPlayer.texture;
then the transition was seamless. So my initial attempt where I inserted yields between the two was the right approach - I just didn't insert quite enough. One short, as a matter of fact. I inserted a counter in the loop, and it showed that I yielded 4 times in the above loop, which is what I would expect, since the video is 30fps, and I'm running at 60fps on my computer. (Sync lock is on.)
A final experiment showed that:
_videoPlayer.Play();
while(_videoPlayer.frame < 1)
{
yield return null;
}
yield return null;
destImage.texture = _videoPlayer.texture;
also did not result in a flicker. (Or, at least, not one that I could see.) So once the VideoPlayer was reporting that it was displaying the second frame (the numbers are 0-based according to the docs), it took one additional game frame before the transition was seamless. (Unless there was a 60-th of a second flicker that my eyes can't see.) That game frame might have something to do with Unity's graphic pipeline or VideoPlayer pipeline - I don't know.
So, the bottom line is that there is a noticeable delay from the time you call Play until there is actually anything in the VideoPlayer's texture that will make it to the screen, and unless you wait for that, you'll be displaying "nothing" (which, in my case, resulted in black background flickering through.)
It occurs to me that since the VideoPlayer is producing a RenderTexture, it might also be possible to blit the previous static texture to the VideoPlayer's texture (so that there would be something there right away) and then do the switch immediately. Another experiment to run...
