The check "left >= INT_MAX - right" is supposed to check for whether left + right does not overflow/wraparound, but given that left and top are uint32_t INT_MAX - right can already wraparound for big values of right (and ordinary 32-bit ints): If right == UINT32_MAX, INT_MAX - right is INT_MAX + 1; for left in 0..par->width both checks will be passed.
Fix this and simplify the check by using 64-bit types, where the addition is guaranteed not to overflow. Signed-off-by: Andreas Rheinhardt <andreas.rheinha...@outlook.com> --- libavformat/matroskaenc.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/libavformat/matroskaenc.c b/libavformat/matroskaenc.c index e1adc0eba6..4b91283119 100644 --- a/libavformat/matroskaenc.c +++ b/libavformat/matroskaenc.c @@ -1786,16 +1786,14 @@ static int mkv_write_track_video(AVFormatContext *s, MatroskaMuxContext *mkv, st->codecpar->nb_coded_side_data, AV_PKT_DATA_FRAME_CROPPING); if (sd && sd->size == sizeof(uint32_t) * 4) { - uint32_t top, bottom, left, right; + uint64_t top, bottom, left, right; top = AV_RL32(sd->data + 0); bottom = AV_RL32(sd->data + 4); left = AV_RL32(sd->data + 8); right = AV_RL32(sd->data + 12); - if (left >= INT_MAX - right || - top >= INT_MAX - bottom || - (left + right) >= par->width || + if ((left + right) >= par->width || (top + bottom) >= par->height) { av_log(s, AV_LOG_ERROR, "Invalid cropping dimensions in stream side data\n"); return AVERROR(EINVAL); -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".