Nuo Mi <nuomi2...@gmail.com> writes: > On Wed, Feb 21, 2024 at 7:10 PM J. Dekker <j...@itanimul.li> wrote: > >> Over/underflow in some cases. >> >> Signed-off-by: J. Dekker <j...@itanimul.li> >> --- >> libavcodec/x86/hevcdsp_init.c | 9 +++++---- >> 1 file changed, 5 insertions(+), 4 deletions(-) >> >> diff --git a/libavcodec/x86/hevcdsp_init.c b/libavcodec/x86/hevcdsp_init.c >> index 31e81eb11f..11cb1b3bfd 100644 >> --- a/libavcodec/x86/hevcdsp_init.c >> +++ b/libavcodec/x86/hevcdsp_init.c >> @@ -1205,10 +1205,11 @@ void ff_hevc_dsp_init_x86(HEVCDSPContext *c, const >> int bit_depth) >> if (EXTERNAL_SSE2(cpu_flags)) { >> c->hevc_v_loop_filter_chroma = >> ff_hevc_v_loop_filter_chroma_12_sse2; >> c->hevc_h_loop_filter_chroma = >> ff_hevc_h_loop_filter_chroma_12_sse2; >> - if (ARCH_X86_64) { >> - c->hevc_v_loop_filter_luma = >> ff_hevc_v_loop_filter_luma_12_sse2; >> - c->hevc_h_loop_filter_luma = >> ff_hevc_h_loop_filter_luma_12_sse2; >> - } >> + // FIXME: 12-bit luma deblock over/underflows in some cases >> + // if (ARCH_X86_64) { >> + // c->hevc_v_loop_filter_luma = >> ff_hevc_v_loop_filter_luma_12_sse2; >> + // c->hevc_h_loop_filter_luma = >> ff_hevc_h_loop_filter_luma_12_sse2; >> + // } >> SAO_BAND_INIT(12, sse2); >> SAO_EDGE_INIT(12, sse2); >> > Hi Dekker, > VVC will utilize this function as well. > Could you please share the HEVC clip or data that caused the overflow? > We'll make efforts to address it during the VVC porting >
You can just run ./tests/checkasm/checkasm --test=hevc_deblock to find a failing case. My guess is that delta0 overflows before the right shift, see the ARM64 asm which specfically widens this calculation on 12 bit variant but I'm not 100%, I don't know x86 asm. -- jd _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".