Re: [FFmpeg-devel] [PATCH] avcodec/celp_math: avoid overflow in shift

2023-09-22 Thread Michael Niedermayer
On Thu, Sep 07, 2023 at 11:24:42PM +0200, Michael Niedermayer wrote: > by making gain unsigned we have 1 bit more available > alternatively we can clip twice as in the g729 reference > > Fixes: left shift of 23404 by 17 places cannot be represented in type 'int' > Fixes: > 61728/clusterfuzz-testc

[FFmpeg-devel] [PATCH] avcodec/celp_math: avoid overflow in shift

2023-09-07 Thread Michael Niedermayer
by making gain unsigned we have 1 bit more available alternatively we can clip twice as in the g729 reference Fixes: left shift of 23404 by 17 places cannot be represented in type 'int' Fixes: 61728/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ACELP_KELVIN_fuzzer-6280412547383296 Found-by: