On Fri, Jun 24, 2016 at 11:17:41AM +0200, Matthieu Bouron wrote: > On Thu, Apr 07, 2016 at 02:51:44PM +0200, Matthieu Bouron wrote: > > On Wed, Mar 23, 2016 at 6:16 PM, Matthieu Bouron <matthieu.bou...@gmail.com> > > wrote: > > > > > > > > > > > On Tue, Mar 22, 2016 at 10:04 AM, Matthieu Bouron < > > > matthieu.bou...@gmail.com> wrote: > > > > > >> > > >> > > >> On Fri, Mar 18, 2016 at 5:50 PM, Matthieu Bouron < > > >> matthieu.bou...@gmail.com> wrote: > > >> > > >>> From: Matthieu Bouron <matthieu.bou...@stupeflix.com> > > >>> > > >>> --- > > >>> > > >>> Hello, > > >>> > > >>> The following patch add hwaccel support to the mediacodec (h264) decoder > > >>> by allowing > > >>> the user to render the output frames directly on a surface. > > >>> > > >>> In order to do so the user needs to initialize the hwaccel through the > > >>> use of > > >>> av_mediacodec_alloc_context and av_mediacodec_default_init functions. > > >>> The later > > >>> takes a reference to an android/view/Surface as parameter. > > >>> > > >>> If the hwaccel successfully initialize, the decoder output frames pix > > >>> fmt will be > > >>> AV_PIX_FMT_MEDIACODEC. The following snippet of code demonstrate how to > > >>> render > > >>> the frames on the surface: > > >>> > > >>> AVMediaCodecBuffer *buffer = (AVMediaCodecBuffer *)frame->data[3]; > > >>> av_mediacodec_release_buffer(buffer, 1); > > >>> > > >>> The last argument of av_mediacodec_release_buffer enable rendering of > > >>> the > > >>> buffer on the surface (or not if set to 0). > > >>> > > >>> Regarding the internal changes in the mediacodec decoder: > > >>> > > >>> MediaCodec.flush() discards both input and output buffers meaning that > > >>> if > > >>> MediaCodec.flush() is called all output buffers the user has a reference > > >>> on are > > >>> now invalid (and cannot be used). > > >>> This behaviour does not fit well in the avcodec API. > > >>> > > >>> When the decoder is configured to output software buffers, there is no > > >>> issue as > > >>> the buffers are copied. > > >>> > > >>> Now when the decoder is configured to output to a surface, the user > > >>> might not > > >>> want to render all the frames as fast as the decoder can go and might > > >>> want to > > >>> control *when* the frame are rendered, so we need to make sure that the > > >>> MediaCodec.flush() call is delayed until all the frames the user retains > > >>> has > > >>> been released or rendered. > > >>> > > >>> Delaying the call to MediaCodec.flush() means buffering any inputs that > > >>> come > > >>> the decoder until the user has released/renderer the frame he retains. > > >>> > > >>> This is a limitation of this hwaccel implementation, if the user retains > > >>> a > > >>> frame (a), then issue a flush command to the decoder, the packets he > > >>> feeds to > > >>> the decoder at that point will be queued in the internal decoder packet > > >>> queue > > >>> (until he releases the frame (a)). This scenario leads to a memory usage > > >>> increase to say the least. > > >>> > > >>> Currently there is no limitation on the size of the internal decoder > > >>> packet > > >>> queue but this is something that can be added easily. Then, if the queue > > >>> is > > >>> full, what would be the behaviour of the decoder ? Can it block ? Or > > >>> should it > > >>> returns something like AVERROR(EAGAIN) ? > > >>> > > >>> About the other internal decoder changes I introduced: > > >>> > > >>> The MediaCodecDecContext is now refcounted (using the lavu/atomic api) > > >>> since > > >>> the (hwaccel) frames can be retained by the user, we need to delay the > > >>> destruction of the codec until the user has released all the frames he > > >>> has a > > >>> reference on. > > >>> The reference counter of the MediaCodecDecContext is incremented each > > >>> time an > > >>> (hwaccel) frame is outputted by the decoder and decremented each time a > > >>> (hwaccel) frame is released. > > >>> > > >>> Also, when the decoder is configured to output to a surface the pts that > > >>> are > > >>> given to the MediaCodec API are now rescaled based on the codec_timebase > > >>> as > > >>> those timestamps values are propagated to the frames rendered on the > > >>> surface > > >>> since Android M. Not sure if it's really useful though. > > >>> > > >>> On the performance side: > > >>> > > >>> On a nexus 5, decoding an h264 stream (main profile) 1080p@60fps: > > >>> - software output + rgba conversion goes at 59~60fps > > >>> - surface output + render on a surface goes at 100~110fps > > >>> > > >>> > > >> [...] > > >> > > >> Patch updated with the following differences: > > >> * the public mediacodec api is now always built (not only when > > >> mediacodec is available) (and the build when mediacodec is not available > > >> has been fixed) > > >> * the documentation of av_mediacodec_release_buffer has been improved a > > >> bit > > >> > > > > > > Patch updated with the following differences: > > > MediaCodecBuffer->released type is now a volatile int (instead of a > > > int*) > > > MediaCodecContext->refcount type is now a volatile int (instead of a > > > int*) > > > > > > > Ping. > > Rebased patch attached. > > Matthieu
> configure | 1 > libavcodec/Makefile | 6 > libavcodec/allcodecs.c | 1 > libavcodec/mediacodec.c | 133 ++++++++++++++++++++ > libavcodec/mediacodec.h | 88 +++++++++++++ > libavcodec/mediacodec_surface.c | 66 +++++++++ > libavcodec/mediacodec_surface.h | 31 ++++ > libavcodec/mediacodec_wrapper.c | 5 > libavcodec/mediacodecdec.c | 265 > +++++++++++++++++++++++++++++++++------- > libavcodec/mediacodecdec.h | 17 ++ > libavcodec/mediacodecdec_h264.c | 44 +++++- > libavcodec/version.h | 2 > libavutil/pixdesc.c | 4 > libavutil/pixfmt.h | 2 > libavutil/version.h | 2 libavutil changes look good, i dont know mediacodec so cannot easily comment about the rest [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Those who are best at talking, realize last or never when they are wrong.
signature.asc
Description: Digital signature
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
