Re: [FFmpeg-devel] [PATCH 1/3] avcodec/x86/vvc/vvc_alf: fix integer overflow

2024-05-29 Thread Ronald S. Bultje
Hi, On Wed, May 29, 2024 at 3:44 PM Wu Jianhua wrote: > Ronald S. Bultje: > > On Wed, May 29, 2024 at 11:38 AM toq...@outlook.com>> wrote: > > +%else > > +vpunpcklqdq m11, m2, m2 > > +vpunpckhqdq m12, m2, m2 > > +vpunpcklwd m11, m11, m14 > > +vpunpcklwd m12

Re: [FFmpeg-devel] [PATCH 1/3] avcodec/x86/vvc/vvc_alf: fix integer overflow

2024-05-29 Thread Ronald S. Bultje
Hi, On Wed, May 29, 2024 at 11:38 AM wrote: > +%else > +vpunpcklqdq m11, m2, m2 > +vpunpckhqdq m12, m2, m2 > +vpunpcklwd m11, m11, m14 > +vpunpcklwd m12, m12, m14 > +paddd m0, m11 > +paddd m1, m12 > +packssdw m0,

Re: [FFmpeg-devel] [PATCH 1/3] avcodec/x86/vvc/vvc_alf: fix integer overflow

2024-05-29 Thread Andreas Rheinhardt
toq...@outlook.com: > From: Wu Jianhua > > Some tests fails with certain seeds > > tests/checkasm/checkasm 2325607578 --test=vvc_alf > checkasm: using random seed 2325607578 > AVX2: > vvc_alf_filter_luma_120x20_12_avx2 (vvc_alf.c:104) > vvc_alf_filter_luma_120x24_12_avx2 (vvc_alf.c:104)