On 13.01.2016 20:06, Michael Niedermayer wrote:
> On Wed, Jan 13, 2016 at 12:52:21AM +0100, Andreas Cadhalpun wrote:
>> ---
>> libavutil/common.h | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/libavutil/common.h b/libavutil/common.h
>> index f3276a2..5ae2847 100644
>> --- a/libavutil/common.h
>> +++ b/libavutil/common.h
>> @@ -211,7 +211,7 @@ static av_always_inline av_const int32_t
>> av_clipl_int32_c(int64_t a)
>> */
>> static av_always_inline av_const int av_clip_intp2_c(int a, int p)
>> {
>> - if ((a + (1 << p)) & ~((2 << p) - 1))
>> + if (a >= INT_MAX - (1 << p) || ((a + (1 << p)) & ~((2 << p) - 1)))
>
> does this differ in any case from using unsigned for the addition?
I don't think so.
> if not then using unsigned avoids the additional operations
OK, patch doing that attached.
Best regards,
Andreas
>From c517f77213d18625cba7ce6b6862a235c4a7c7d2 Mon Sep 17 00:00:00 2001
From: Andreas Cadhalpun <andreas.cadhal...@googlemail.com>
Date: Thu, 14 Jan 2016 01:15:22 +0100
Subject: [PATCH] lavu: prevent overflow in av_clip_intp2_c
This fixes ubsan runtime error: signed integer overflow: 8388608 +
2140274688 cannot be represented in type 'int'
Signed-off-by: Andreas Cadhalpun <andreas.cadhal...@googlemail.com>
---
libavutil/common.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/libavutil/common.h b/libavutil/common.h
index f9766ad..7b7bcbe 100644
--- a/libavutil/common.h
+++ b/libavutil/common.h
@@ -211,7 +211,7 @@ static av_always_inline av_const int32_t av_clipl_int32_c(int64_t a)
*/
static av_always_inline av_const int av_clip_intp2_c(int a, int p)
{
- if ((a + (1 << p)) & ~((2 << p) - 1))
+ if (((unsigned)a + (1 << p)) & ~((2 << p) - 1))
return (a >> 31) ^ ((1 << p) - 1);
else
return a;
--
2.6.4
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
