> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-boun...@ffmpeg.org> On Behalf Of Chema > Gonzalez > Sent: 2022年11月12日 0:55 > To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org> > Cc: matthieu.bou...@stupeflix.com; a...@tmm1.net > Subject: Re: [FFmpeg-devel] [Internet][PATCH 00/12] Add MediaCodec encoder > and NDK MediaCodec support > > On Thu, Nov 10, 2022 at 7:36 PM "zhilizhao(赵志立)" <quinkbl...@foxmail.com> > wrote: > > > > Ping for review. > > > > > On Oct 24, 2022, at 11:16, Zhao Zhili <quinkbl...@foxmail.com> wrote: > > > > > > From: Zhao Zhili <zhiliz...@tencent.com> > > > > > > Firstly, some bugs were fixed (patch 1-4). > > > > > > Patch 5 and 6 make mediacodec_wrapper support Java MediaCodec and NDK > > > MediaCodec. The use case I'm considering is run FFmpeg on cmdline without > > > JVM, > > > for example, run FFmpeg inside of termux (an Android terminal emulator). > > > It's > > > well known that NDK MediaCodec missing some important functions, like get > > > the > > > list of codecs, but still useable. > > > > > > Patch 7 add NDK MediaCodec decoder support. It can be enabled via options, > > > and enabled automatically if no JVM is available. > > > > > > Patch 8 add ANativeWindow support to hwcontext_mediacodec. It can be set > > > by > > > user, and can be created via AMediaCodec_createPersistentInputSurface > > > automatically. This is a preparation for encoder. > > > > > > Patch 9 makes MediaCodec decoder to support ANativeWindow directly. It > > > worth > > > to note that AVMediaCodecContext has only surface. Although we provided > > > av_mediacodec_alloc_context(), we didn't strictly prevent users to > > > allocate > > > AVMediaCodecContext on stack. I'm not sure if it's OK to add new field to > > > AVMediaCodecContext. > > > > > > Patch 10 add MediaCodec encoder support. Frame can be feed to encoder via > > > buffer, or via Surface/ANativeWindow. If Surface/ANativeWindow is used, > > > and > > > the frames come from our MediaCodec decoder wrapper, we can control it's > > > 'render' (send to encoder's surface) via av_mediacodec_release_buffer(). > > > A DTS > > > generation strategy works in this case. However, if frames comes from > > > other > > > sources, like a camera, there is no way to control the 'render' yet, so > > > DTS is > > > missing in this case. > > > > > > Finally, we can do mediacodec transcoding with FFmpeg cmdline on Android. > > > More importantly, we can do MediaCodec decoder to encoder without copy > > > frames, > > > although it's very limited since most of avfilters doesn't work. For > > > example: > > > > > > ./ffmpeg -hwaccel mediacodec -hwaccel_output_format mediacodec -i > > > /sdcard/test.mp4 -an -c:v h264_mediacodec -y > /sdcard/out.mp4 > > How do you build the ffmpeg binary you're using for experiments in the > android device?
Just configure with --enable-mediacodec and --enable-jni. Here is the build script I used for the development. https://github.com/quink-black/ffmpeg-ci/blob/master/android_ffmpeg.sh Here is the Android Studio project which can be used to test the mediacodec wrapper implemented via JNI. > > > > > > > > > > Since there is no real AVHWFrameContext implementation in > > > hwcontext_mediacodec. > > > there is no hwframe_ctx for mediacodec and av_hwframe_transfer_data() > > > doesn't > > > work. So if -hwaccel_output_format isn't being specified like: > > > > > > ./ffmpeg -hwaccel mediacodec -i /sdcard/test.mp4 -an -c:v h264_mediacodec > > > -y /sdcard/out.mp4 > > > > > > It will trigger a crash in av_hwframe_transfer_data. Patch 11 add a check > > > on > > > hwframe_ctx. Patch 12 set hwaccel_output_format automatically to avoid > > > such > > > case. > > > > > > > > > Zhao Zhili (12): > > > avcodec/mediacodec: fix incorrect crop info > > > avcodec/mediacodecdec: don't break out if both input and output port > > > return try again > > > avcodec/mediacodecdec_common: fix misuse av_free/av_freep > > > avcodec/mediacodecdec_common: fix useless av_buffer_unref > > > avcodec/mediacodec_wrapper: separate implementation from interface > > > avcodec/mediacodec: add NDK media codec wrapper > > > avcodec/mediacodecdec: enable NDK mediacodec > > > avutil/hwcontext_mediacodec: add ANativeWindow support > > > avcodec/mediacodec: add ANativeWindow support > > > avcodec: add MediaCodec encoder > > > avutil/hwcontext: verify hw_frames_ctx in transfer_data_alloc > > > fftools/ffmpeg_opt: set default hwaccel_output_format for mediacodec > > > > > > Changelog | 2 + > > > configure | 6 + > > > fftools/ffmpeg_opt.c | 4 + > > > libavcodec/Makefile | 2 + > > > libavcodec/allcodecs.c | 2 + > > > libavcodec/mediacodec_surface.c | 46 +- > > > libavcodec/mediacodec_surface.h | 8 +- > > > libavcodec/mediacodec_wrapper.c | 942 +++++++++++++++++++++++++++--- > > > libavcodec/mediacodec_wrapper.h | 275 +++++++-- > > > libavcodec/mediacodecdec.c | 21 +- > > > libavcodec/mediacodecdec_common.c | 35 +- > > > libavcodec/mediacodecdec_common.h | 1 + > > > libavcodec/mediacodecenc.c | 495 ++++++++++++++++ > > > libavcodec/version.h | 2 +- > > > libavutil/hwcontext.c | 6 +- > > > libavutil/hwcontext_mediacodec.c | 56 +- > > > libavutil/hwcontext_mediacodec.h | 11 + > > > libavutil/version.h | 4 +- > > > 18 files changed, 1780 insertions(+), 138 deletions(-) > > > create mode 100644 libavcodec/mediacodecenc.c > > > > > > -- > > > 2.25.1 > > > > > > > > > > _______________________________________________ > > ffmpeg-devel mailing list > > ffmpeg-devel@ffmpeg.org > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > To unsubscribe, visit link above, or email > > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
