PR #20733 opened by mkver URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20733 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20733.patch
Uncovered by UBSan when running the mpegvideoencdsp checkasm test. Will apply this tonight unless there are objections. >From 05b8608c76e76b7d8b4ce5e86e0940244fbb737e Mon Sep 17 00:00:00 2001 From: Andreas Rheinhardt <[email protected]> Date: Tue, 21 Oct 2025 12:11:55 +0200 Subject: [PATCH] avcodec/x86/mpegvideoencdsp_init: Fix left shift of negative number Uncovered by UBSan when running the mpegvideoencdsp checkasm test. Signed-off-by: Andreas Rheinhardt <[email protected]> --- libavcodec/x86/mpegvideoencdsp_init.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavcodec/x86/mpegvideoencdsp_init.c b/libavcodec/x86/mpegvideoencdsp_init.c index 3cd16fefbf..bf5b722016 100644 --- a/libavcodec/x86/mpegvideoencdsp_init.c +++ b/libavcodec/x86/mpegvideoencdsp_init.c @@ -42,7 +42,7 @@ static int try_8x8basis_ssse3(const int16_t rem[64], const int16_t weight[64], c x86_reg i=0; av_assert2(FFABS(scale) < MAX_ABS); - scale <<= 16 + SCALE_OFFSET - BASIS_SHIFT + RECON_SHIFT; + scale *= 1 << (16 + SCALE_OFFSET - BASIS_SHIFT + RECON_SHIFT); __asm__ volatile( "pxor %%xmm2, %%xmm2 \n\t" @@ -87,7 +87,7 @@ static void add_8x8basis_ssse3(int16_t rem[64], const int16_t basis[64], int sca x86_reg i=0; if (FFABS(scale) < 1024) { - scale <<= 16 + SCALE_OFFSET - BASIS_SHIFT + RECON_SHIFT; + scale *= 1 << (16 + SCALE_OFFSET - BASIS_SHIFT + RECON_SHIFT); __asm__ volatile( "movd %3, %%xmm2 \n\t" "punpcklwd %%xmm2, %%xmm2 \n\t" -- 2.49.1 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
