> -----Original Message----- > From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On Behalf > Of Fu, Linjie > Sent: Friday, November 16, 2018 16:37 > To: FFmpeg development discussions and patches <ffmpeg- > de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a > specific decode format > > > -----Original Message----- > > From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On > Behalf > > Of Mark Thompson > > Sent: Thursday, November 15, 2018 05:48 > > To: ffmpeg-devel@ffmpeg.org > > Subject: Re: [FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a > > specific decode format > > > > On 14/11/18 01:35, Fu, Linjie wrote: > > >> -----Original Message----- > > >> From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On > > Behalf > > >> Of Mark Thompson > > >> Sent: Wednesday, November 14, 2018 09:11 > > >> To: ffmpeg-devel@ffmpeg.org > > >> Subject: Re: [FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a > > >> specific decode format > > >> > > >> On 14/11/18 00:50, Fu, Linjie wrote: > > >>>> -----Original Message----- > > >>>> From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On > > >> Behalf > > >>>> Of Mark Thompson > > >>>> Sent: Wednesday, November 14, 2018 07:44 > > >>>> To: ffmpeg-devel@ffmpeg.org > > >>>> Subject: [FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a > > >> specific > > >>>> decode format > > >>>> > > >>>> Fixes #7519. > > >>>> --- > > >>>> doc/ffmpeg.texi | 13 ++++++++++++ > > >>>> fftools/ffmpeg.c | 10 ++++++++++ > > >>>> fftools/ffmpeg.h | 4 ++++ > > >>>> fftools/ffmpeg_opt.c | 47 > > >>>> ++++++++++++++++++++++++++++++++++++++++++++ > > >>>> 4 files changed, 74 insertions(+) > > >>>> > > >>>> diff --git a/doc/ffmpeg.texi b/doc/ffmpeg.texi > > >>>> index 3717f22d42..d127bc0f0d 100644 > > >>>> --- a/doc/ffmpeg.texi > > >>>> +++ b/doc/ffmpeg.texi > > >>>> @@ -920,6 +920,19 @@ would be more efficient. > > >>>> When doing stream copy, copy also non-key frames found at the > > >>>> beginning. > > >>>> > > >>>> +@item -decode_format[:@var{stream_specifier}] > > >>>> @var{pixfmt}[,@var{pixfmt}...] (@emph{input,per-stream}) > > >>>> +Set the possible output formats to be used by the decoder for this > > >> stream. > > >>>> +If the decoder does not natively support any format in the given list > > for > > >>>> +the input stream then decoding will fail rather than continuing with a > > >>>> +different format. > > >>>> + > > >>>> +In general this should not be set - the decoder will select an output > > >>>> +format based on the input stream parameters and available > > components, > > >>>> and > > >>>> +that will be automatically converted to whatever the output > requires. > > It > > >>>> +may be useful to force a hardware decoder supporting output in > > >> multiple > > >>>> +different memory types to pick a desired one, or to ensure that a > > >> hardware > > >>>> +decoder is used when software fallback is also available. > > >>>> + > > >>>> @item -init_hw_device > > >>>> @var{type}[=@var{name}][:@var{device}[,@var{key=value}...]] > > >>>> Initialise a new hardware device of type @var{type} called > > @var{name}, > > >>>> using the > > >>>> given device parameters. > > >>>> diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c > > >>>> index 38c21e944a..c651c8d3a8 100644 > > >>>> --- a/fftools/ffmpeg.c > > >>>> +++ b/fftools/ffmpeg.c > > >>>> @@ -598,6 +598,7 @@ static void ffmpeg_cleanup(int ret) > > >>>> avsubtitle_free(&ist->prev_sub.subtitle); > > >>>> av_frame_free(&ist->sub2video.frame); > > >>>> av_freep(&ist->filters); > > >>>> + av_freep(&ist->decode_formats); > > >>>> av_freep(&ist->hwaccel_device); > > >>>> av_freep(&ist->dts_buffer); > > >>>> > > >>>> @@ -2800,6 +2801,15 @@ static enum AVPixelFormat > > >>>> get_format(AVCodecContext *s, const enum AVPixelFormat > > >>>> const AVCodecHWConfig *config = NULL; > > >>>> int i; > > >>>> > > >>>> + if (ist->decode_formats) { > > >>>> + for (i = 0; ist->decode_formats[i] != AV_PIX_FMT_NONE; > > >>>> i++) > { > > >>>> + if (ist->decode_formats[i] == *p) > > >>>> + break; > > >>>> + } > > >>>> + if (ist->decode_formats[i] != *p) > > >>>> + continue; > > >>>> + } > > >>>> + > > >>>> if (!(desc->flags & AV_PIX_FMT_FLAG_HWACCEL)) > > >>>> break; > > >>>> > > >>>> diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h > > >>>> index eb1eaf6363..b06fd18b1c 100644 > > >>>> --- a/fftools/ffmpeg.h > > >>>> +++ b/fftools/ffmpeg.h > > >>>> @@ -125,6 +125,8 @@ typedef struct OptionsContext { > > >>>> int nb_ts_scale; > > >>>> SpecifierOpt *dump_attachment; > > >>>> int nb_dump_attachment; > > >>>> + SpecifierOpt *decode_formats; > > >>>> + int nb_decode_formats; > > >>>> SpecifierOpt *hwaccels; > > >>>> int nb_hwaccels; > > >>>> SpecifierOpt *hwaccel_devices; > > >>>> @@ -334,6 +336,8 @@ typedef struct InputStream { > > >>>> int top_field_first; > > >>>> int guess_layout_max; > > >>>> > > >>>> + enum AVPixelFormat *decode_formats; > > >>>> + > > >>>> int autorotate; > > >>>> > > >>>> int fix_sub_duration; > > >>>> diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c > > >>>> index d4851a2cd8..63bb05053b 100644 > > >>>> --- a/fftools/ffmpeg_opt.c > > >>>> +++ b/fftools/ffmpeg_opt.c > > >>>> @@ -701,6 +701,7 @@ static void > add_input_streams(OptionsContext > > *o, > > >>>> AVFormatContext *ic) > > >>>> AVStream *st = ic->streams[i]; > > >>>> AVCodecParameters *par = st->codecpar; > > >>>> InputStream *ist = av_mallocz(sizeof(*ist)); > > >>>> + char *decode_formats = NULL; > > >>>> char *framerate = NULL, *hwaccel_device = NULL; > > >>>> const char *hwaccel = NULL; > > >>>> char *hwaccel_output_format = NULL; > > >>>> @@ -797,6 +798,49 @@ static void > add_input_streams(OptionsContext > > >> *o, > > >>>> AVFormatContext *ic) > > >>>> ist->top_field_first = -1; > > >>>> MATCH_PER_STREAM_OPT(top_field_first, i, ist- > >top_field_first, > > ic, > > >>>> st); > > >>>> > > >>>> + MATCH_PER_STREAM_OPT(decode_formats, str, > > >> decode_formats, ic, > > >>>> st); > > >>>> + if (decode_formats) { > > >>>> + const char *p, *q; > > >>>> + int i, nb_formats; > > >>>> + char tmp[32]; > > >>>> + > > >>>> + nb_formats = 0; > > >>>> + for (p = decode_formats; p; p = strchr(p + 1, ',')) > > >>>> + ++nb_formats; > > >>>> + > > >>>> + ist->decode_formats = > > >>>> + av_malloc_array(nb_formats + 1, sizeof(*ist- > > >>> decode_formats)); > > >>>> + if (!ist->decode_formats) > > >>>> + exit_program(1); > > >>>> + > > >>>> + p = decode_formats; > > >>>> + for (i = 0; i < nb_formats; i++) { > > >>>> + q = strchr(p, ','); > > >>>> + if (!q) { > > >>>> + ist->decode_formats[i] = av_get_pix_fmt(p); > > >>>> + if (ist->decode_formats[i] == AV_PIX_FMT_NONE) > > >>>> + break; > > >>>> + continue; > > >>>> + } > > >>>> + if (q - p > sizeof(tmp) - 1) > > >>>> + break; > > >>>> + memcpy(tmp, p, q - p); > > >>>> + tmp[q - p] = 0; > > >>>> + ist->decode_formats[i] = av_get_pix_fmt(tmp); > > >>>> + if (ist->decode_formats[i] == AV_PIX_FMT_NONE) > > >>>> + break; > > >>>> + p = q + 1; > > >>>> + } > > >>>> + if (i < nb_formats) { > > >>>> + av_log(NULL, AV_LOG_FATAL, > > >>>> + "Unrecognised decode format: %s.\n", p); > > >>>> + exit_program(1); > > >>>> + } > > >>>> + ist->decode_formats[nb_formats] = AV_PIX_FMT_NONE; > > >>>> + } else { > > >>>> + ist->decode_formats = NULL; > > >>>> + } > > >>>> + > > >>>> MATCH_PER_STREAM_OPT(hwaccels, str, hwaccel, ic, st); > > >>>> if (hwaccel) { > > >>>> // The NVDEC hwaccels use a CUDA device, so remap the > > name > > >> here. > > >>>> @@ -3583,6 +3627,9 @@ const OptionDef options[] = { > > >>>> "audio bitrate (please use -b:a)", "bitrate" }, > > >>>> { "b", OPT_VIDEO | HAS_ARG | OPT_PERFILE | OPT_OUTPUT, > > >>>> { .func_arg = opt_bitrate }, > > >>>> "video bitrate (please use -b:v)", "bitrate" }, > > >>>> + { "decode_format", OPT_VIDEO | OPT_STRING | HAS_ARG | > > >>>> OPT_EXPERT | > > >>>> + OPT_SPEC | OPT_INPUT, > > >>>> { .off = > > >>>> OFFSET(decode_formats) }, > > >>>> + "set output format(s) to be used by decoder, fail if none of > these > > >>>> formats are available", "format" }, > > >>>> { "hwaccel", OPT_VIDEO | OPT_STRING | HAS_ARG | > > OPT_EXPERT > > >> | > > >>>> OPT_SPEC | OPT_INPUT, > > >>>> { .off = > > >>>> OFFSET(hwaccels) }, > > >>>> "use HW accelerated decoding", "hwaccel name" }, > > >>>> -- > > >>>> 2.19.1 > > >>>> > > >>> > > >>> Is there any other conditions which will cause the hwaccel failed and > > >> fallback to software path? > > >>> Like profile was not supported by hardware? > > >> > > >> If this option is set then get_format will never choose a format not on > the > > list > > >> you provide, so it shouldn't be able to use software decode unless a > > >> software format is on the list. > > >> > > >>> I applied the patch and tested some cases, like yuv422 VAAPI > decoding > > >> (currently not supported), HEVC decoding with unsupported profile. > > >>> It reports an error like "hwaccel initialization returned error", and > > continue > > >> to decode and produce less frames than expected. (which I thought > > fallback > > >> still happened). > > >> > > >> Can you give an example? With -v debug there will be output saying > > exactly > > >> what happened in ff_get_format(). > > >> > > >> - Mark > > > > > > > > > Got information like this: > > > > > > [hevc @ 0x55b87457dbc0] Unknown HEVC profile: 0 > > > [hevc @ 0x55b87457dbc0] Decoding SPS > > > [hevc @ 0x55b87457dbc0] Unknown HEVC profile: 0 > > > [hevc @ 0x55b87457dbc0] Decoding PPS > > > [hevc @ 0x55b87457dbc0] Format vaapi_vld chosen by get_format(). > > > [hevc @ 0x55b87457dbc0] Format vaapi_vld requires hwaccel initialisation. > > > [hevc @ 0x55b87457dbc0] Codec hevc profile 0 not supported for > hardware > > decode. > > > [hevc @ 0x55b87457dbc0] Failed setup for format vaapi_vld: hwaccel > > initialisation returned error. > > > [hevc @ 0x55b87457dbc0] Format vaapi_vld not usable, retrying > > get_format() without it. > > > [hevc @ 0x55b87457dbc0] Error parsing NAL unit #3. > > > cur_dts is invalid (this is harmless if it occurs once at the start per > > > stream) > > > [hevc @ 0x55b87458b7c0] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, > > temporal_id: 0 > > > [hevc @ 0x55b87458b7c0] Could not find ref with POC 0 > > > > > > See the attached report file for more details. > > > > This is a decoder bug - get_format() is not correctly called again after it > > fails, > > and the decoder then uses the software pixfmt derived from the SPS on > > subsequent frames. > > > > (I did pretty much all of my testing with H.264, which does not have this > > problem.) > > > Works for me on forcing hwaccel in HEVC vaapi decoding after applying the > hevcdec patch. > Thanks for answering my questions. > Ping? Can this be merged? https://trac.ffmpeg.org/ticket/7519#comment:4
--- Linjie _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
