On Wed, 2 Nov 2022, Michael Niedermayer wrote:
On Wed, Nov 02, 2022 at 10:16:57PM +0100, Andreas Rheinhardt wrote:
Michael Niedermayer:
On Wed, Nov 02, 2022 at 10:02:39PM +0100, Michael Niedermayer wrote:
Fixes: integer overflow
Signed-off-by: Michael Niedermayer <mich...@niedermayer.cc>
---
libswscale/output.c | 25 +++++++++++--------------
libswscale/x86/output.asm | 16 +++++++++++++++-
2 files changed, 26 insertions(+), 15 deletions(-)
Note, these corner case overflows could affect some of the similary
implemented cases that are not depth 16 too
ill fix them if issues are replicated
The checkasm-sw_gbrp runs into many overflows (when run under UBSan);
e.g. fate.ffmpeg.org tells me of an issue in line 2289. Said line is not
touched in your commits.
checkasm-sw_gbrp feeds random data widely outside sane ranges in.
the test certainly makes no sense for testing asm. There is no point
in matching C for widely invalid cases. Of cousre we shouldnt overflow
if any of this can be triggered with valid and real input (which probably
can be done in some cases)
Patches for the checkasm tests, to make them produce sane input values in
the correct ranges, are very much welcome!
// Martin
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
