On 18/02/2019 05:05, Fu, Linjie wrote: >> -----Original Message----- >> From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On Behalf >> Of Fu, Linjie >> Sent: Friday, November 16, 2018 16:37 >> To: FFmpeg development discussions and patches <ffmpeg- >> de...@ffmpeg.org> >> Subject: Re: [FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a >> specific decode format >> >>> -----Original Message----- >>> From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On >> Behalf >>> Of Mark Thompson >>> Sent: Thursday, November 15, 2018 05:48 >>> To: ffmpeg-devel@ffmpeg.org >>> Subject: Re: [FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a >>> specific decode format >>> >>> On 14/11/18 01:35, Fu, Linjie wrote: >>>>> -----Original Message----- >>>>> From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On >>> Behalf >>>>> Of Mark Thompson >>>>> Sent: Wednesday, November 14, 2018 09:11 >>>>> To: ffmpeg-devel@ffmpeg.org >>>>> Subject: Re: [FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a >>>>> specific decode format >>>>> >>>>> On 14/11/18 00:50, Fu, Linjie wrote: >>>>>>> -----Original Message----- >>>>>>> From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On >>>>> Behalf >>>>>>> Of Mark Thompson >>>>>>> Sent: Wednesday, November 14, 2018 07:44 >>>>>>> To: ffmpeg-devel@ffmpeg.org >>>>>>> Subject: [FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a >>>>> specific >>>>>>> decode format >>>>>>> >>>>>>> Fixes #7519. >>>>>>> --- >>>>>>> doc/ffmpeg.texi | 13 ++++++++++++ >>>>>>> fftools/ffmpeg.c | 10 ++++++++++ >>>>>>> fftools/ffmpeg.h | 4 ++++ >>>>>>> fftools/ffmpeg_opt.c | 47 >>>>>>> ++++++++++++++++++++++++++++++++++++++++++++ >>>>>>> 4 files changed, 74 insertions(+) >>>>>>> >>>>>>> diff --git a/doc/ffmpeg.texi b/doc/ffmpeg.texi >>>>>>> index 3717f22d42..d127bc0f0d 100644 >>>>>>> --- a/doc/ffmpeg.texi >>>>>>> +++ b/doc/ffmpeg.texi >>>>>>> @@ -920,6 +920,19 @@ would be more efficient. >>>>>>> When doing stream copy, copy also non-key frames found at the >>>>>>> beginning. >>>>>>> >>>>>>> +@item -decode_format[:@var{stream_specifier}] >>>>>>> @var{pixfmt}[,@var{pixfmt}...] (@emph{input,per-stream}) >>>>>>> +Set the possible output formats to be used by the decoder for this >>>>> stream. >>>>>>> +If the decoder does not natively support any format in the given list >>> for >>>>>>> +the input stream then decoding will fail rather than continuing with a >>>>>>> +different format. >>>>>>> + >>>>>>> +In general this should not be set - the decoder will select an output >>>>>>> +format based on the input stream parameters and available >>> components, >>>>>>> and >>>>>>> +that will be automatically converted to whatever the output >> requires. >>> It >>>>>>> +may be useful to force a hardware decoder supporting output in >>>>> multiple >>>>>>> +different memory types to pick a desired one, or to ensure that a >>>>> hardware >>>>>>> +decoder is used when software fallback is also available. >>>>>>> + >>>>>>> @item -init_hw_device >>>>>>> @var{type}[=@var{name}][:@var{device}[,@var{key=value}...]] >>>>>>> Initialise a new hardware device of type @var{type} called >>> @var{name}, >>>>>>> using the >>>>>>> given device parameters. >>>>>>> diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c >>>>>>> index 38c21e944a..c651c8d3a8 100644 >>>>>>> --- a/fftools/ffmpeg.c >>>>>>> +++ b/fftools/ffmpeg.c >>>>>>> @@ -598,6 +598,7 @@ static void ffmpeg_cleanup(int ret) >>>>>>> avsubtitle_free(&ist->prev_sub.subtitle); >>>>>>> av_frame_free(&ist->sub2video.frame); >>>>>>> av_freep(&ist->filters); >>>>>>> + av_freep(&ist->decode_formats); >>>>>>> av_freep(&ist->hwaccel_device); >>>>>>> av_freep(&ist->dts_buffer); >>>>>>> >>>>>>> @@ -2800,6 +2801,15 @@ static enum AVPixelFormat >>>>>>> get_format(AVCodecContext *s, const enum AVPixelFormat >>>>>>> const AVCodecHWConfig *config = NULL; >>>>>>> int i; >>>>>>> >>>>>>> + if (ist->decode_formats) { >>>>>>> + for (i = 0; ist->decode_formats[i] != AV_PIX_FMT_NONE; i++) >> { >>>>>>> + if (ist->decode_formats[i] == *p) >>>>>>> + break; >>>>>>> + } >>>>>>> + if (ist->decode_formats[i] != *p) >>>>>>> + continue; >>>>>>> + } >>>>>>> + >>>>>>> if (!(desc->flags & AV_PIX_FMT_FLAG_HWACCEL)) >>>>>>> break; >>>>>>> >>>>>>> diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h >>>>>>> index eb1eaf6363..b06fd18b1c 100644 >>>>>>> --- a/fftools/ffmpeg.h >>>>>>> +++ b/fftools/ffmpeg.h >>>>>>> @@ -125,6 +125,8 @@ typedef struct OptionsContext { >>>>>>> int nb_ts_scale; >>>>>>> SpecifierOpt *dump_attachment; >>>>>>> int nb_dump_attachment; >>>>>>> + SpecifierOpt *decode_formats; >>>>>>> + int nb_decode_formats; >>>>>>> SpecifierOpt *hwaccels; >>>>>>> int nb_hwaccels; >>>>>>> SpecifierOpt *hwaccel_devices; >>>>>>> @@ -334,6 +336,8 @@ typedef struct InputStream { >>>>>>> int top_field_first; >>>>>>> int guess_layout_max; >>>>>>> >>>>>>> + enum AVPixelFormat *decode_formats; >>>>>>> + >>>>>>> int autorotate; >>>>>>> >>>>>>> int fix_sub_duration; >>>>>>> diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c >>>>>>> index d4851a2cd8..63bb05053b 100644 >>>>>>> --- a/fftools/ffmpeg_opt.c >>>>>>> +++ b/fftools/ffmpeg_opt.c >>>>>>> @@ -701,6 +701,7 @@ static void >> add_input_streams(OptionsContext >>> *o, >>>>>>> AVFormatContext *ic) >>>>>>> AVStream *st = ic->streams[i]; >>>>>>> AVCodecParameters *par = st->codecpar; >>>>>>> InputStream *ist = av_mallocz(sizeof(*ist)); >>>>>>> + char *decode_formats = NULL; >>>>>>> char *framerate = NULL, *hwaccel_device = NULL; >>>>>>> const char *hwaccel = NULL; >>>>>>> char *hwaccel_output_format = NULL; >>>>>>> @@ -797,6 +798,49 @@ static void >> add_input_streams(OptionsContext >>>>> *o, >>>>>>> AVFormatContext *ic) >>>>>>> ist->top_field_first = -1; >>>>>>> MATCH_PER_STREAM_OPT(top_field_first, i, ist- >>> top_field_first, >>> ic, >>>>>>> st); >>>>>>> >>>>>>> + MATCH_PER_STREAM_OPT(decode_formats, str, >>>>> decode_formats, ic, >>>>>>> st); >>>>>>> + if (decode_formats) { >>>>>>> + const char *p, *q; >>>>>>> + int i, nb_formats; >>>>>>> + char tmp[32]; >>>>>>> + >>>>>>> + nb_formats = 0; >>>>>>> + for (p = decode_formats; p; p = strchr(p + 1, ',')) >>>>>>> + ++nb_formats; >>>>>>> + >>>>>>> + ist->decode_formats = >>>>>>> + av_malloc_array(nb_formats + 1, sizeof(*ist- >>>>>> decode_formats)); >>>>>>> + if (!ist->decode_formats) >>>>>>> + exit_program(1); >>>>>>> + >>>>>>> + p = decode_formats; >>>>>>> + for (i = 0; i < nb_formats; i++) { >>>>>>> + q = strchr(p, ','); >>>>>>> + if (!q) { >>>>>>> + ist->decode_formats[i] = av_get_pix_fmt(p); >>>>>>> + if (ist->decode_formats[i] == AV_PIX_FMT_NONE) >>>>>>> + break; >>>>>>> + continue; >>>>>>> + } >>>>>>> + if (q - p > sizeof(tmp) - 1) >>>>>>> + break; >>>>>>> + memcpy(tmp, p, q - p); >>>>>>> + tmp[q - p] = 0; >>>>>>> + ist->decode_formats[i] = av_get_pix_fmt(tmp); >>>>>>> + if (ist->decode_formats[i] == AV_PIX_FMT_NONE) >>>>>>> + break; >>>>>>> + p = q + 1; >>>>>>> + } >>>>>>> + if (i < nb_formats) { >>>>>>> + av_log(NULL, AV_LOG_FATAL, >>>>>>> + "Unrecognised decode format: %s.\n", p); >>>>>>> + exit_program(1); >>>>>>> + } >>>>>>> + ist->decode_formats[nb_formats] = AV_PIX_FMT_NONE; >>>>>>> + } else { >>>>>>> + ist->decode_formats = NULL; >>>>>>> + } >>>>>>> + >>>>>>> MATCH_PER_STREAM_OPT(hwaccels, str, hwaccel, ic, st); >>>>>>> if (hwaccel) { >>>>>>> // The NVDEC hwaccels use a CUDA device, so remap the >>> name >>>>> here. >>>>>>> @@ -3583,6 +3627,9 @@ const OptionDef options[] = { >>>>>>> "audio bitrate (please use -b:a)", "bitrate" }, >>>>>>> { "b", OPT_VIDEO | HAS_ARG | OPT_PERFILE | OPT_OUTPUT, >>>>>>> { .func_arg = opt_bitrate }, >>>>>>> "video bitrate (please use -b:v)", "bitrate" }, >>>>>>> + { "decode_format", OPT_VIDEO | OPT_STRING | HAS_ARG | >>>>>>> OPT_EXPERT | >>>>>>> + OPT_SPEC | OPT_INPUT, >>>>>>> { .off = >>>>>>> OFFSET(decode_formats) }, >>>>>>> + "set output format(s) to be used by decoder, fail if none of >> these >>>>>>> formats are available", "format" }, >>>>>>> { "hwaccel", OPT_VIDEO | OPT_STRING | HAS_ARG | >>> OPT_EXPERT >>>>> | >>>>>>> OPT_SPEC | OPT_INPUT, >>>>>>> { .off = >>>>>>> OFFSET(hwaccels) }, >>>>>>> "use HW accelerated decoding", "hwaccel name" }, >>>>>>> -- >>>>>>> 2.19.1 >>>>>>> >>>>>> >>>>>> Is there any other conditions which will cause the hwaccel failed and >>>>> fallback to software path? >>>>>> Like profile was not supported by hardware? >>>>> >>>>> If this option is set then get_format will never choose a format not on >> the >>> list >>>>> you provide, so it shouldn't be able to use software decode unless a >>>>> software format is on the list. >>>>> >>>>>> I applied the patch and tested some cases, like yuv422 VAAPI >> decoding >>>>> (currently not supported), HEVC decoding with unsupported profile. >>>>>> It reports an error like "hwaccel initialization returned error", and >>> continue >>>>> to decode and produce less frames than expected. (which I thought >>> fallback >>>>> still happened). >>>>> >>>>> Can you give an example? With -v debug there will be output saying >>> exactly >>>>> what happened in ff_get_format(). >>>>> >>>>> - Mark >>>> >>>> >>>> Got information like this: >>>> >>>> [hevc @ 0x55b87457dbc0] Unknown HEVC profile: 0 >>>> [hevc @ 0x55b87457dbc0] Decoding SPS >>>> [hevc @ 0x55b87457dbc0] Unknown HEVC profile: 0 >>>> [hevc @ 0x55b87457dbc0] Decoding PPS >>>> [hevc @ 0x55b87457dbc0] Format vaapi_vld chosen by get_format(). >>>> [hevc @ 0x55b87457dbc0] Format vaapi_vld requires hwaccel initialisation. >>>> [hevc @ 0x55b87457dbc0] Codec hevc profile 0 not supported for >> hardware >>> decode. >>>> [hevc @ 0x55b87457dbc0] Failed setup for format vaapi_vld: hwaccel >>> initialisation returned error. >>>> [hevc @ 0x55b87457dbc0] Format vaapi_vld not usable, retrying >>> get_format() without it. >>>> [hevc @ 0x55b87457dbc0] Error parsing NAL unit #3. >>>> cur_dts is invalid (this is harmless if it occurs once at the start per >>>> stream) >>>> [hevc @ 0x55b87458b7c0] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, >>> temporal_id: 0 >>>> [hevc @ 0x55b87458b7c0] Could not find ref with POC 0 >>>> >>>> See the attached report file for more details. >>> >>> This is a decoder bug - get_format() is not correctly called again after it >>> fails, >>> and the decoder then uses the software pixfmt derived from the SPS on >>> subsequent frames. >>> >>> (I did pretty much all of my testing with H.264, which does not have this >>> problem.) >>> >> Works for me on forcing hwaccel in HEVC vaapi decoding after applying the >> hevcdec patch. >> Thanks for answering my questions. >> > Ping? > Can this be merged? > https://trac.ffmpeg.org/ticket/7519#comment:4
I think this got lost because I was feeling it loosely depended on <https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2018-November/236376.html> in order to actually work in the intended use-case. I wasn't particularly confident in that patch being the best way to do it and it isn't my code, so I was hoping for any thoughts from someone more familiar with it. Shall I apply this part anyway? Thanks, - Mark _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
