Question by enio_orqafpv · Jul 23, 2021 at 02:36 PM · streaming stream libraries live

Decode H.264 stream and display in Unity

I'm trying to do raw decoding of H.264 stream, whose packages I'm getting from an active UDP connection those are groups of 4 packages which are making an buffer which is representing one video frame. I don't know much about H.264 decoding but I'm trying to have input which is accepting mentioned buffer from UDP connection, decoding it and applying it to the texture inside Unity.

I tried following things: - Vive Media Decoder asset (https://assetstore.unity.com/packages/tools/video/vive-media-decoder-63938), it's doing great job while working with protocols like rtmp, rtsp etc. but for raw decoding from incoming buffer can't do much as it's wrapped around ffmpeg and other decoding tools - C written H.264 decoder (https://github.com/shengbinmeng/ffmpeg-h264-dec), depends on many libav libraries which I'm unable to track and adjust and later than convert into .dll which will be accessible from C# inside Unity - C++ written H.264 decoder (https://www.roxlu.com/2014/039/decoding-h264-and-yuv420p-playback), similar as above I'm getting compile time errors for unkown functions which again depends on libav/ffmpeg libraries - C# written H.264 decoder (https://github.com/pansk/cscodec), seems to be working with url's but still not sure if I could send buffer direct to it and not sure how to include whole visual studio c# project into unity project and make it work - Also I have looked at https://github.com/Ruslan-B/FFmpeg.AutoGen but was unable to find a way to solve my problem

Basiclly, what I'm looking for is an written .dll (C/C++ program) file which is accepting H.264 encoded frames (buffers) and returning RGB array, which is used and called inside Unity project.

Add comment · Show 2