On Fri, 2021-07-30 at 08:18 +0000, Soft Works wrote: > > -----Original Message----- > > From: ffmpeg-devel <ffmpeg-devel-boun...@ffmpeg.org> On Behalf Of > > Haihao Xiang > > Sent: Thursday, 29 July 2021 09:04 > > To: ffmpeg-devel@ffmpeg.org > > Cc: Haihao Xiang <haihao.xi...@intel.com> > > Subject: [FFmpeg-devel] [PATCH v2 1/2] qsvdec: add support for > > HW_DEVICE_CTX method > > > > This allows user set hw_device_ctx instead of hw_frames_ctx for QSV > > decoders, hence we may remove the ad-hoc libmfx setup code from > > FFmpeg. > > > > "-hwaccel_output_format format" is applied to QSV decoders after removing > > the ad-hoc libmfx code. To keep compatibility with old commandlines, the > > default format is set to AV_PIX_FMT_QSV. User should set "- > > hwaccel_output_format qsv" explicitly if AV_PIX_FMT_QSV is expected. > > > > User may use the normal way to set device for QSV decoders now. > > "-qsv_device device" is deprecated and will be removed from FFmpeg. > > > > For example: > > > > $> ffmpeg -init_hw_device vaapi=va:/dev/dri/card0 -init_hw_device > > qsv=hw@va -hwaccel qsv -c:v h264_qsv -i input.h264 -f null - > > Hi Haihao, > > Why so complicated with double device initialization? > The regular way is this: > > ffmpeg -init_hw_device "qsv=dev:hw_any,child_device=/dev/dri/card0" -hwaccel > qsv -c:v h264_qsv -i input.h264 -f null - > > Or for Windows: > > ffmpeg -init_hw_device "qsv=dev:hw_any,child_device=1" -hwaccel qsv -c:v > h264_qsv -i input.h264 -f null -
After applying this patch, the above commands may work too, actually a child device is created internally in this way, then a QSV device is derived from the child device implicitly. However deriving a device explicitly creates a connection between source device and derived device. We may derive a new device of the source type from the derived device. e.g. The command below works $> ffmpeg -init_hw_device vaapi=intel:/dev/dri/renderD128 -init_hw_device qsv=qsv@intel -hwaccel qsv -c:v h264_qsv -i input.h264 -vf hwmap=derive_device=vaapi,scale_vaapi -f null - but the command below doesn't work $> ffmpeg -init_hw_device qsv=dev:hw_any,child_device=/dev/dri/renderD128 -hwaccel qsv -c:v h264_qsv -i ~/graphics/movies/720p.mp4 -vf hwmap=derive_device=vaapi,scale_vaapi -f null - (We may apply http://ffmpeg.org/pipermail/ffmpeg-devel/2021-February/276695.html to create a connection between two devices too, but this change breaks 'make fate-hwdevice V=2'). > > > > > /dev/dri/renderD128 is actually open for h264_qsv decoder without this > > patch. After applying this patch, /dev/dri/card0 is used. > > --- > > v2: Rebase the patchset against the latest master branch and resolve a > > conflict. > > > > fftools/Makefile | 1 - > > fftools/ffmpeg.h | 1 - > > fftools/ffmpeg_hw.c | 12 +++++ > > fftools/ffmpeg_opt.c | 54 +++++++++++++++++++-- fftools/ffmpeg_qsv.c > > > 109 ------------------------------------------- > > > > libavcodec/qsvdec.c | 31 +++++++++++- > > 6 files changed, 92 insertions(+), 116 deletions(-) delete mode 100644 > > fftools/ffmpeg_qsv.c > > > > diff --git a/fftools/Makefile b/fftools/Makefile index > > 5affaa3f56..5234932ab0 > > 100644 > > --- a/fftools/Makefile > > +++ b/fftools/Makefile > > @@ -10,7 +10,6 @@ ALLAVPROGS = > > $(AVBASENAMES:%=%$(PROGSSUF)$(EXESUF)) > > ALLAVPROGS_G = $(AVBASENAMES:%=%$(PROGSSUF)_g$(EXESUF)) > > > > OBJS-ffmpeg += fftools/ffmpeg_opt.o > > fftools/ffmpeg_filter.o > > fftools/ffmpeg_hw.o > > -OBJS-ffmpeg-$(CONFIG_LIBMFX) += fftools/ffmpeg_qsv.o > > ifndef CONFIG_VIDEOTOOLBOX > > OBJS-ffmpeg-$(CONFIG_VDA) += fftools/ffmpeg_videotoolbox.o > > endif > > diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index > > d9c0628657..d2dd7ca092 100644 > > --- a/fftools/ffmpeg.h > > +++ b/fftools/ffmpeg.h > > @@ -61,7 +61,6 @@ enum HWAccelID { > > HWACCEL_AUTO, > > HWACCEL_GENERIC, > > HWACCEL_VIDEOTOOLBOX, > > - HWACCEL_QSV, > > }; > > > > typedef struct HWAccel { > > diff --git a/fftools/ffmpeg_hw.c b/fftools/ffmpeg_hw.c index > > fc4a5d31d6..043e4c5863 100644 > > --- a/fftools/ffmpeg_hw.c > > +++ b/fftools/ffmpeg_hw.c > > @@ -339,6 +339,18 @@ int hw_device_setup_for_decode(InputStream *ist) > > } else if (ist->hwaccel_id == HWACCEL_GENERIC) { > > type = ist->hwaccel_device_type; > > dev = hw_device_get_by_type(type); > > + > > + // When "-qsv_device device" is used, an internal QSV device > > named > > + // as "__qsv_device" is created and another QSV device is > > created > > + // if "-init_hw_device qsv=name@name" is used. There are 2 QSV > > devices > > + // if both "-qsv_device device" and "-init_hw_device > > qsv=name@name" > > + // are used, hw_device_get_by_type(AV_HWDEVICE_TYPE_QSV) > > returns NULL. > > + // To keep back-compatibility with the removed ad-hoc libmfx > > setup > > code, > > + // call hw_device_get_by_name("__qsv_device") to select the > > internal QSV > > + // device. This will be removed once -qsv_device is no longer > > supported. > > + if (!dev && type == AV_HWDEVICE_TYPE_QSV) > > + dev = hw_device_get_by_name("__qsv_device"); > > + > > if (!dev) > > err = hw_device_init_from_type(type, NULL, &dev); > > } else { > > diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index > > 1b43bab9fc..4ddc6c939b 100644 > > --- a/fftools/ffmpeg_opt.c > > +++ b/fftools/ffmpeg_opt.c > > @@ -137,9 +137,6 @@ static const char *const opt_name_enc_time_bases[] > > = {"enc_time_base" > > const HWAccel hwaccels[] = { > > #if CONFIG_VIDEOTOOLBOX > > { "videotoolbox", videotoolbox_init, HWACCEL_VIDEOTOOLBOX, > > AV_PIX_FMT_VIDEOTOOLBOX }, -#endif -#if CONFIG_LIBMFX > > - { "qsv", qsv_init, HWACCEL_QSV, AV_PIX_FMT_QSV }, > > #endif > > { 0 }, > > }; > > @@ -571,6 +568,49 @@ static int opt_vaapi_device(void *optctx, const char > > *opt, const char *arg) } #endif > > > > +#if CONFIG_QSV > > +static int opt_qsv_device(void *optctx, const char *opt, const char > > +*arg) { #if CONFIG_VAAPI > > + const char *prefix = "vaapi=__qsv_child_device:"; #elif > > +CONFIG_DXVA2 > > + const char *prefix = "dxva2=__qsv_child_device:"; #else > > + const char *prefix = NULL; > > +#endif > > Same like for the command line. You can directly initialize a QSV context > with a single operation. According to the above explanation, I prefer to derive device explicitly. > > [...] > > > + if (avctx->hw_device_ctx && !avctx->hw_frames_ctx && ret == > > AV_PIX_FMT_QSV) { > > + AVHWFramesContext *hwframes_ctx; > > + AVQSVFramesContext *frames_hwctx; > > + > > + avctx->hw_frames_ctx = > > + av_hwframe_ctx_alloc(avctx->hw_device_ctx); > > + > > + if (!avctx->hw_frames_ctx) { > > + av_log(avctx, AV_LOG_ERROR, "av_hwframe_ctx_alloc failed\n"); > > + return AVERROR(ENOMEM); > > + } > > + > > + hwframes_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx- > > > data; > > > > + frames_hwctx = hwframes_ctx->hwctx; > > + hwframes_ctx->width = FFALIGN(avctx->coded_width, 32); > > + hwframes_ctx->height = FFALIGN(avctx->coded_height, 32); > > + hwframes_ctx->format = AV_PIX_FMT_QSV; > > + hwframes_ctx->sw_format = avctx->sw_pix_fmt; > > + hwframes_ctx->initial_pool_size = 64 + avctx->extra_hw_frames; > > > I'm not sure whether 64 is a good default choice. I had to reduce this to 32 > when processing 4k video with OpenCL interop (memory sharing), otherwise I got > OOM errors. (with video mem set to the maximum of 1G in the BIOS) > 64 is used in ffmpeg_qsv.c, we have another patchset to optimize the memory usage in qsv path and will submit it once this patchset is merged. > > Besides the above comments, this looks good to me in general. It is the right > step to go forward IMO. > > Just the double device initialization should not be required (it isn't now) > when it comes to the selection between D3D9 and D3D11. I am simply using a > device parameter for this, and a command line looks like this: > > ffmpeg -init_hw_device "qsv=dev:hw_any,child_device=1,dx11=1" -hwaccel qsv > -c:v h264_qsv -i input.h264 -f null - > According to the above explanation, you can still use this way for the selection between D3D9 and D3D11. Thanks Haihao > > But let's get this done done first.. > softworkz > > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
