*bump* > -----Original Message----- > From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On Behalf Of Mark > Thompson > Sent: Wednesday, February 20, 2019 1:52 PM > To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a specific > decode format > > On 18/02/2019 05:05, Fu, Linjie wrote: > >> -----Original Message----- > >> From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On Behalf > >> Of Fu, Linjie > >> Sent: Friday, November 16, 2018 16:37 > >> To: FFmpeg development discussions and patches <ffmpeg- > >> de...@ffmpeg.org> > >> Subject: Re: [FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a > >> specific decode format > >> > >>> -----Original Message----- > >>> From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On > >> Behalf > >>> Of Mark Thompson > >>> Sent: Thursday, November 15, 2018 05:48 > >>> To: ffmpeg-devel@ffmpeg.org > >>> Subject: Re: [FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a > >>> specific decode format > >>> > >>> On 14/11/18 01:35, Fu, Linjie wrote: > >>>>> -----Original Message----- > >>>>> From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On > >>> Behalf > >>>>> Of Mark Thompson > >>>>> Sent: Wednesday, November 14, 2018 09:11 > >>>>> To: ffmpeg-devel@ffmpeg.org > >>>>> Subject: Re: [FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a > >>>>> specific decode format > >>>>> > >>>>> On 14/11/18 00:50, Fu, Linjie wrote: > >>>>>>> -----Original Message----- > >>>>>>> From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On > >>>>> Behalf > >>>>>>> Of Mark Thompson > >>>>>>> Sent: Wednesday, November 14, 2018 07:44 > >>>>>>> To: ffmpeg-devel@ffmpeg.org > >>>>>>> Subject: [FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a > >>>>> specific > >>>>>>> decode format > >>>>>>> > >>>>>>> Fixes #7519. > >>>>>>> --- > >>>>>>> doc/ffmpeg.texi | 13 ++++++++++++ > >>>>>>> fftools/ffmpeg.c | 10 ++++++++++ > >>>>>>> fftools/ffmpeg.h | 4 ++++ > >>>>>>> fftools/ffmpeg_opt.c | 47 > >>>>>>> ++++++++++++++++++++++++++++++++++++++++++++ > >>>>>>> 4 files changed, 74 insertions(+) > >>>>>>> > >>>>>>> diff --git a/doc/ffmpeg.texi b/doc/ffmpeg.texi > >>>>>>> index 3717f22d42..d127bc0f0d 100644 > >>>>>>> --- a/doc/ffmpeg.texi > >>>>>>> +++ b/doc/ffmpeg.texi > >>>>>>> @@ -920,6 +920,19 @@ would be more efficient. > >>>>>>> When doing stream copy, copy also non-key frames found at the > >>>>>>> beginning. > >>>>>>> > >>>>>>> +@item -decode_format[:@var{stream_specifier}] > >>>>>>> @var{pixfmt}[,@var{pixfmt}...] (@emph{input,per-stream}) > >>>>>>> +Set the possible output formats to be used by the decoder for this > >>>>> stream. > >>>>>>> +If the decoder does not natively support any format in the given list > >>> for > >>>>>>> +the input stream then decoding will fail rather than continuing with > >>>>>>> a > >>>>>>> +different format. > >>>>>>> + > >>>>>>> +In general this should not be set - the decoder will select an output > >>>>>>> +format based on the input stream parameters and available > >>> components, > >>>>>>> and > >>>>>>> +that will be automatically converted to whatever the output > >> requires. > >>> It > >>>>>>> +may be useful to force a hardware decoder supporting output in > >>>>> multiple > >>>>>>> +different memory types to pick a desired one, or to ensure that a > >>>>> hardware > >>>>>>> +decoder is used when software fallback is also available. > >>>>>>> + > >>>>>>> @item -init_hw_device > >>>>>>> @var{type}[=@var{name}][:@var{device}[,@var{key=value}...]] > >>>>>>> Initialise a new hardware device of type @var{type} called > >>> @var{name}, > >>>>>>> using the > >>>>>>> given device parameters. > >>>>>>> diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c > >>>>>>> index 38c21e944a..c651c8d3a8 100644 > >>>>>>> --- a/fftools/ffmpeg.c > >>>>>>> +++ b/fftools/ffmpeg.c > >>>>>>> @@ -598,6 +598,7 @@ static void ffmpeg_cleanup(int ret) > >>>>>>> avsubtitle_free(&ist->prev_sub.subtitle); > >>>>>>> av_frame_free(&ist->sub2video.frame); > >>>>>>> av_freep(&ist->filters); > >>>>>>> + av_freep(&ist->decode_formats); > >>>>>>> av_freep(&ist->hwaccel_device); > >>>>>>> av_freep(&ist->dts_buffer); > >>>>>>> > >>>>>>> @@ -2800,6 +2801,15 @@ static enum AVPixelFormat > >>>>>>> get_format(AVCodecContext *s, const enum AVPixelFormat > >>>>>>> const AVCodecHWConfig *config = NULL; > >>>>>>> int i; > >>>>>>> > >>>>>>> + if (ist->decode_formats) { > >>>>>>> + for (i = 0; ist->decode_formats[i] != AV_PIX_FMT_NONE; > >>>>>>> i++) > >> { > >>>>>>> + if (ist->decode_formats[i] == *p) > >>>>>>> + break; > >>>>>>> + } > >>>>>>> + if (ist->decode_formats[i] != *p) > >>>>>>> + continue; > >>>>>>> + } > >>>>>>> + > >>>>>>> if (!(desc->flags & AV_PIX_FMT_FLAG_HWACCEL)) > >>>>>>> break; > >>>>>>> > >>>>>>> diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h > >>>>>>> index eb1eaf6363..b06fd18b1c 100644 > >>>>>>> --- a/fftools/ffmpeg.h > >>>>>>> +++ b/fftools/ffmpeg.h > >>>>>>> @@ -125,6 +125,8 @@ typedef struct OptionsContext { > >>>>>>> int nb_ts_scale; > >>>>>>> SpecifierOpt *dump_attachment; > >>>>>>> int nb_dump_attachment; > >>>>>>> + SpecifierOpt *decode_formats; > >>>>>>> + int nb_decode_formats; > >>>>>>> SpecifierOpt *hwaccels; > >>>>>>> int nb_hwaccels; > >>>>>>> SpecifierOpt *hwaccel_devices; > >>>>>>> @@ -334,6 +336,8 @@ typedef struct InputStream { > >>>>>>> int top_field_first; > >>>>>>> int guess_layout_max; > >>>>>>> > >>>>>>> + enum AVPixelFormat *decode_formats; > >>>>>>> + > >>>>>>> int autorotate; > >>>>>>> > >>>>>>> int fix_sub_duration; > >>>>>>> diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c > >>>>>>> index d4851a2cd8..63bb05053b 100644 > >>>>>>> --- a/fftools/ffmpeg_opt.c > >>>>>>> +++ b/fftools/ffmpeg_opt.c > >>>>>>> @@ -701,6 +701,7 @@ static void > >> add_input_streams(OptionsContext > >>> *o, > >>>>>>> AVFormatContext *ic) > >>>>>>> AVStream *st = ic->streams[i]; > >>>>>>> AVCodecParameters *par = st->codecpar; > >>>>>>> InputStream *ist = av_mallocz(sizeof(*ist)); > >>>>>>> + char *decode_formats = NULL; > >>>>>>> char *framerate = NULL, *hwaccel_device = NULL; > >>>>>>> const char *hwaccel = NULL; > >>>>>>> char *hwaccel_output_format = NULL; > >>>>>>> @@ -797,6 +798,49 @@ static void > >> add_input_streams(OptionsContext > >>>>> *o, > >>>>>>> AVFormatContext *ic) > >>>>>>> ist->top_field_first = -1; > >>>>>>> MATCH_PER_STREAM_OPT(top_field_first, i, ist- > >>> top_field_first, > >>> ic, > >>>>>>> st); > >>>>>>> > >>>>>>> + MATCH_PER_STREAM_OPT(decode_formats, str, > >>>>> decode_formats, ic, > >>>>>>> st); > >>>>>>> + if (decode_formats) { > >>>>>>> + const char *p, *q; > >>>>>>> + int i, nb_formats; > >>>>>>> + char tmp[32]; > >>>>>>> + > >>>>>>> + nb_formats = 0; > >>>>>>> + for (p = decode_formats; p; p = strchr(p + 1, ',')) > >>>>>>> + ++nb_formats; > >>>>>>> + > >>>>>>> + ist->decode_formats = > >>>>>>> + av_malloc_array(nb_formats + 1, sizeof(*ist- > >>>>>> decode_formats)); > >>>>>>> + if (!ist->decode_formats) > >>>>>>> + exit_program(1); > >>>>>>> + > >>>>>>> + p = decode_formats; > >>>>>>> + for (i = 0; i < nb_formats; i++) { > >>>>>>> + q = strchr(p, ','); > >>>>>>> + if (!q) { > >>>>>>> + ist->decode_formats[i] = av_get_pix_fmt(p); > >>>>>>> + if (ist->decode_formats[i] == > >>>>>>> AV_PIX_FMT_NONE) > >>>>>>> + break; > >>>>>>> + continue; > >>>>>>> + } > >>>>>>> + if (q - p > sizeof(tmp) - 1) > >>>>>>> + break; > >>>>>>> + memcpy(tmp, p, q - p); > >>>>>>> + tmp[q - p] = 0; > >>>>>>> + ist->decode_formats[i] = av_get_pix_fmt(tmp); > >>>>>>> + if (ist->decode_formats[i] == AV_PIX_FMT_NONE) > >>>>>>> + break; > >>>>>>> + p = q + 1; > >>>>>>> + } > >>>>>>> + if (i < nb_formats) { > >>>>>>> + av_log(NULL, AV_LOG_FATAL, > >>>>>>> + "Unrecognised decode format: %s.\n", p); > >>>>>>> + exit_program(1); > >>>>>>> + } > >>>>>>> + ist->decode_formats[nb_formats] = AV_PIX_FMT_NONE; > >>>>>>> + } else { > >>>>>>> + ist->decode_formats = NULL; > >>>>>>> + } > >>>>>>> + > >>>>>>> MATCH_PER_STREAM_OPT(hwaccels, str, hwaccel, ic, st); > >>>>>>> if (hwaccel) { > >>>>>>> // The NVDEC hwaccels use a CUDA device, so remap the > >>> name > >>>>> here. > >>>>>>> @@ -3583,6 +3627,9 @@ const OptionDef options[] = { > >>>>>>> "audio bitrate (please use -b:a)", "bitrate" }, > >>>>>>> { "b", OPT_VIDEO | HAS_ARG | OPT_PERFILE | OPT_OUTPUT, > >>>>>>> { .func_arg = opt_bitrate }, > >>>>>>> "video bitrate (please use -b:v)", "bitrate" }, > >>>>>>> + { "decode_format", OPT_VIDEO | OPT_STRING | HAS_ARG | > >>>>>>> OPT_EXPERT | > >>>>>>> + OPT_SPEC | OPT_INPUT, > >>>>>>> { .off = > >>>>>>> OFFSET(decode_formats) }, > >>>>>>> + "set output format(s) to be used by decoder, fail if none of > >> these > >>>>>>> formats are available", "format" }, > >>>>>>> { "hwaccel", OPT_VIDEO | OPT_STRING | HAS_ARG | > >>> OPT_EXPERT > >>>>> | > >>>>>>> OPT_SPEC | OPT_INPUT, > >>>>>>> { .off = > >>>>>>> OFFSET(hwaccels) }, > >>>>>>> "use HW accelerated decoding", "hwaccel name" }, > >>>>>>> -- > >>>>>>> 2.19.1 > >>>>>>> > >>>>>> > >>>>>> Is there any other conditions which will cause the hwaccel failed and > >>>>> fallback to software path? > >>>>>> Like profile was not supported by hardware? > >>>>> > >>>>> If this option is set then get_format will never choose a format not on > >> the > >>> list > >>>>> you provide, so it shouldn't be able to use software decode unless a > >>>>> software format is on the list. > >>>>> > >>>>>> I applied the patch and tested some cases, like yuv422 VAAPI > >> decoding > >>>>> (currently not supported), HEVC decoding with unsupported profile. > >>>>>> It reports an error like "hwaccel initialization returned error", and > >>> continue > >>>>> to decode and produce less frames than expected. (which I thought > >>> fallback > >>>>> still happened). > >>>>> > >>>>> Can you give an example? With -v debug there will be output saying > >>> exactly > >>>>> what happened in ff_get_format(). > >>>>> > >>>>> - Mark > >>>> > >>>> > >>>> Got information like this: > >>>> > >>>> [hevc @ 0x55b87457dbc0] Unknown HEVC profile: 0 > >>>> [hevc @ 0x55b87457dbc0] Decoding SPS > >>>> [hevc @ 0x55b87457dbc0] Unknown HEVC profile: 0 > >>>> [hevc @ 0x55b87457dbc0] Decoding PPS > >>>> [hevc @ 0x55b87457dbc0] Format vaapi_vld chosen by get_format(). > >>>> [hevc @ 0x55b87457dbc0] Format vaapi_vld requires hwaccel initialisation. > >>>> [hevc @ 0x55b87457dbc0] Codec hevc profile 0 not supported for > >> hardware > >>> decode. > >>>> [hevc @ 0x55b87457dbc0] Failed setup for format vaapi_vld: hwaccel > >>> initialisation returned error. > >>>> [hevc @ 0x55b87457dbc0] Format vaapi_vld not usable, retrying > >>> get_format() without it. > >>>> [hevc @ 0x55b87457dbc0] Error parsing NAL unit #3. > >>>> cur_dts is invalid (this is harmless if it occurs once at the start per > >>>> stream) > >>>> [hevc @ 0x55b87458b7c0] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, > >>> temporal_id: 0 > >>>> [hevc @ 0x55b87458b7c0] Could not find ref with POC 0 > >>>> > >>>> See the attached report file for more details. > >>> > >>> This is a decoder bug - get_format() is not correctly called again after > >>> it fails, > >>> and the decoder then uses the software pixfmt derived from the SPS on > >>> subsequent frames. > >>> > >>> (I did pretty much all of my testing with H.264, which does not have this > >>> problem.) > >>> > >> Works for me on forcing hwaccel in HEVC vaapi decoding after applying the > >> hevcdec patch. > >> Thanks for answering my questions. > >> > > Ping? > > Can this be merged? > > https://trac.ffmpeg.org/ticket/7519#comment:4 > > I think this got lost because I was feeling it loosely depended on > <https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2018- > November/236376.html> in order to actually work in the intended use-case. I > wasn't particularly confident in that patch being the > best way to do it and it isn't my code, so I was hoping for any thoughts from > someone more familiar with it. > > Shall I apply this part anyway? > > Thanks, > > - Mark > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
