Hi! This is my first contribution to the project so please excuse any bad etiquette, I tried to read all the FAQs before posting. Would love to start by thanking everyone for such an amazing framework you've built!
Anyway, here's my proposed patch to support video resizing when using NVDEC hwaccel to decode hevc video (I could look into a similar patch for h264, av1, etc if this looks useful). There's a bit more context/explanation in the commit description in the patch, but please let me know if the use case isn't clear. I tested locally and all of these 4 scenarios work as expected: * Using hevc codec with nvdec hwaccel, leaving avctx->width and avctx->height unset. On a 1920x1080 input video, I get 1920x1080 cuda frames out. * Using hevc codec with nvdec hwaccel, setting avctx->width and avctx->height to some arbitrary value (e.g. 640x360). On the same input video, I get 640x360 cuda frames out. * Using hevc codec without hwaccel, leaving avctx->width and avctx->height unset. I get 1920x1080 yuvj420p frames (in cpu) out. * Using hevc codec without hwaccel, setting avctx->width and avctx->height to some arbitrary value (e.g. 640x360). The values get ignored (as in FFMpeg master) and I again get 1920x1080 yuvj420p frames out. I'm not extremely familiar with hevcdec.c so I'm not sure if this would accidentally break something else. Looking forward to hearing your thoughts! >From 850afda5f6479064c75a4b905f12e48f97b6d551 Mon Sep 17 00:00:00 2001 From: Carlos Ruiz <carlos.r.do...@gmail.com> Date: Thu, 19 Sep 2024 14:00:05 +0200 Subject: [PATCH] avcodec/nvdec: support resizing while decoding Nvidia chips support accelerated resizing while decoding video. The *_cuvid codecs (cuviddec.c) already support resizing and cropping, but have two big downsides: 1) they have a minimum latency of two packets (even with the LOW_DELAY flag enabled) 2) AV_CODEC_FLAG_COPY_OPAQUE is not respected (opaque and opaque_ref aren't transferred from packets to frames) Instead, parsing the video using a non-accelerated codec (hevcdec.c) solves both downsides above. This commit brings resizing capabilities to the *_nvdec hwaccel, similar to what *_cuvid does, to combine the best of both worlds (proper parsing + accelerated decoding and resizing). --- libavcodec/hevc/hevcdec.c | 8 ++++++-- libavcodec/nvdec.c | 21 +++++++++++++++++---- 2 files changed, 23 insertions(+), 6 deletions(-) diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index d915d74d22..d63fc5875f 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -351,8 +351,12 @@ static void export_stream_params(HEVCContext *s, const HEVCSPS *sps) avctx->pix_fmt = sps->pix_fmt; avctx->coded_width = sps->width; avctx->coded_height = sps->height; - avctx->width = sps->width - ow->left_offset - ow->right_offset; - avctx->height = sps->height - ow->top_offset - ow->bottom_offset; + if (avctx->width <= 0 || avctx->height <= 0) { + avctx->width = sps->width; + avctx->height = sps->height; + } + avctx->width = avctx->width - ow->left_offset - ow->right_offset; + avctx->height = avctx->height - ow->top_offset - ow->bottom_offset; avctx->has_b_frames = sps->temporal_layer[sps->max_sub_layers - 1].num_reorder_pics; avctx->profile = sps->ptl.general_ptl.profile_idc; avctx->level = sps->ptl.general_ptl.level_idc; diff --git a/libavcodec/nvdec.c b/libavcodec/nvdec.c index 932544564a..86143de74c 100644 --- a/libavcodec/nvdec.c +++ b/libavcodec/nvdec.c @@ -324,6 +324,18 @@ static int nvdec_init_hwframes(AVCodecContext *avctx, AVBufferRef **out_frames_r return 0; } +static int get_buffer2(AVCodecContext *avctx, AVFrame *frame, int flags) { + /* + * HEVC codec includes FF_CODEC_CAP_EXPORTS_CROPPING in its caps_internal, so by default frames will be set + * to width=avctx->coded_width and height=avctx->coded_height. Now that we support resizing as part of decoding, + * overwrite the frame dimensions with display values rather than coded. + */ + int ret = avcodec_default_get_buffer2(avctx, frame, flags); + frame->width = avctx->width; + frame->height = avctx->height; + return ret; +} + int ff_nvdec_decode_init(AVCodecContext *avctx) { NVDECContext *ctx = avctx->internal->hwaccel_priv_data; @@ -393,8 +405,9 @@ int ff_nvdec_decode_init(AVCodecContext *avctx) params.ulWidth = avctx->coded_width; params.ulHeight = avctx->coded_height; - params.ulTargetWidth = avctx->coded_width; - params.ulTargetHeight = avctx->coded_height; + avctx->get_buffer2 = get_buffer2; + params.ulTargetWidth = avctx->width; + params.ulTargetHeight = avctx->height; params.bitDepthMinus8 = sw_desc->comp[0].depth - 8; params.OutputFormat = output_format; params.CodecType = cuvid_codec_type; @@ -719,8 +732,8 @@ int ff_nvdec_frame_params(AVCodecContext *avctx, chroma_444 = supports_444 && cuvid_chroma_format == cudaVideoChromaFormat_444; frames_ctx->format = AV_PIX_FMT_CUDA; - frames_ctx->width = (avctx->coded_width + 1) & ~1; - frames_ctx->height = (avctx->coded_height + 1) & ~1; + frames_ctx->width = (avctx->width + 1) & ~1; + frames_ctx->height = (avctx->height + 1) & ~1; /* * We add two extra frames to the pool to account for deinterlacing filters * holding onto their frames. -- 2.43.0 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".