On Wed, Mar 23, 2016 at 6:16 PM, Matthieu Bouron <matthieu.bou...@gmail.com> wrote:
> > > On Tue, Mar 22, 2016 at 10:04 AM, Matthieu Bouron < > matthieu.bou...@gmail.com> wrote: > >> >> >> On Fri, Mar 18, 2016 at 5:50 PM, Matthieu Bouron < >> matthieu.bou...@gmail.com> wrote: >> >>> From: Matthieu Bouron <matthieu.bou...@stupeflix.com> >>> >>> --- >>> >>> Hello, >>> >>> The following patch add hwaccel support to the mediacodec (h264) decoder >>> by allowing >>> the user to render the output frames directly on a surface. >>> >>> In order to do so the user needs to initialize the hwaccel through the >>> use of >>> av_mediacodec_alloc_context and av_mediacodec_default_init functions. >>> The later >>> takes a reference to an android/view/Surface as parameter. >>> >>> If the hwaccel successfully initialize, the decoder output frames pix >>> fmt will be >>> AV_PIX_FMT_MEDIACODEC. The following snippet of code demonstrate how to >>> render >>> the frames on the surface: >>> >>> AVMediaCodecBuffer *buffer = (AVMediaCodecBuffer *)frame->data[3]; >>> av_mediacodec_release_buffer(buffer, 1); >>> >>> The last argument of av_mediacodec_release_buffer enable rendering of the >>> buffer on the surface (or not if set to 0). >>> >>> Regarding the internal changes in the mediacodec decoder: >>> >>> MediaCodec.flush() discards both input and output buffers meaning that if >>> MediaCodec.flush() is called all output buffers the user has a reference >>> on are >>> now invalid (and cannot be used). >>> This behaviour does not fit well in the avcodec API. >>> >>> When the decoder is configured to output software buffers, there is no >>> issue as >>> the buffers are copied. >>> >>> Now when the decoder is configured to output to a surface, the user >>> might not >>> want to render all the frames as fast as the decoder can go and might >>> want to >>> control *when* the frame are rendered, so we need to make sure that the >>> MediaCodec.flush() call is delayed until all the frames the user retains >>> has >>> been released or rendered. >>> >>> Delaying the call to MediaCodec.flush() means buffering any inputs that >>> come >>> the decoder until the user has released/renderer the frame he retains. >>> >>> This is a limitation of this hwaccel implementation, if the user retains >>> a >>> frame (a), then issue a flush command to the decoder, the packets he >>> feeds to >>> the decoder at that point will be queued in the internal decoder packet >>> queue >>> (until he releases the frame (a)). This scenario leads to a memory usage >>> increase to say the least. >>> >>> Currently there is no limitation on the size of the internal decoder >>> packet >>> queue but this is something that can be added easily. Then, if the queue >>> is >>> full, what would be the behaviour of the decoder ? Can it block ? Or >>> should it >>> returns something like AVERROR(EAGAIN) ? >>> >>> About the other internal decoder changes I introduced: >>> >>> The MediaCodecDecContext is now refcounted (using the lavu/atomic api) >>> since >>> the (hwaccel) frames can be retained by the user, we need to delay the >>> destruction of the codec until the user has released all the frames he >>> has a >>> reference on. >>> The reference counter of the MediaCodecDecContext is incremented each >>> time an >>> (hwaccel) frame is outputted by the decoder and decremented each time a >>> (hwaccel) frame is released. >>> >>> Also, when the decoder is configured to output to a surface the pts that >>> are >>> given to the MediaCodec API are now rescaled based on the codec_timebase >>> as >>> those timestamps values are propagated to the frames rendered on the >>> surface >>> since Android M. Not sure if it's really useful though. >>> >>> On the performance side: >>> >>> On a nexus 5, decoding an h264 stream (main profile) 1080p@60fps: >>> - software output + rgba conversion goes at 59~60fps >>> - surface output + render on a surface goes at 100~110fps >>> >>> >> [...] >> >> Patch updated with the following differences: >> * the public mediacodec api is now always built (not only when >> mediacodec is available) (and the build when mediacodec is not available >> has been fixed) >> * the documentation of av_mediacodec_release_buffer has been improved a >> bit >> > > Patch updated with the following differences: > MediaCodecBuffer->released type is now a volatile int (instead of a int*) > MediaCodecContext->refcount type is now a volatile int (instead of a > int*) > Ping. _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
