On 01.07.2024 18:52, Josh Allmann wrote:
On Thu, 20 Jun 2024 at 17:39, Josh Allmann <joshua.allm...@gmail.com> wrote:In intra-only mode, frameIntervalP is 0, which means the frame data array is smaller than the number of surfaces. This causes a crash when closing the encoder. Fix this by making sure the frame data array is at least as big as the number of surfaces. --- libavcodec/nvenc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c index a9945355ba..93e87b21db 100644 --- a/libavcodec/nvenc.c +++ b/libavcodec/nvenc.c @@ -1021,7 +1021,7 @@ static av_cold int nvenc_recalc_surfaces(AVCodecContext *avctx) // Output in the worst case will only start when the surface buffer is completely full. // Hence we need to keep at least the max amount of surfaces plus the max reorder delay around. - ctx->frame_data_array_nb = ctx->nb_surfaces + ctx->encode_config.frameIntervalP - 1; + ctx->frame_data_array_nb = FFMAX(ctx->nb_surfaces, ctx->nb_surfaces + ctx->encode_config.frameIntervalP - 1); return 0; } -- 2.39.2Hello, Ping for review. This patch fixes an easily triggered crash with nvenc in intra-only mode, eg ffmpeg -i <in> -c:v h264_nvenc -g 0 <out> Josh _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
It's the wrong fix for the crash, but definitely a sensible change in itself. Will amend with the actual fix (using the right size when deallocating that array) but keep this change as well, since it makes sense.
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
