Derp, actually, 2 << 63 doesn't fit in int64_t either, this check should be < 63. Fixed.
On Fri, Nov 17, 2017 at 1:38 PM, Dale Curtis <dalecur...@chromium.org> wrote: > 2 << (wrap_bits=64 - 1) does not fit in int64_t; apply the check > used in other places that handle wrap bits to ensure the values > are <= 63. > > Signed-off-by: Dale Curtis <dalecur...@chromium.org> > >
From 4ae4992326487ba0e42fa7fcf2a53fe3d4564780 Mon Sep 17 00:00:00 2001 From: Dale Curtis <dalecur...@chromium.org> Date: Fri, 17 Nov 2017 13:35:56 -0800 Subject: [PATCH] [avformat] Prevent undefined shift with wrap_bits >= 63. 2 << (wrap_bits=63 - 1) does not fit in int64_t; apply the check used in other places that handle wrap bits to ensure the values are < 63. Signed-off-by: Dale Curtis <dalecur...@chromium.org> --- libavformat/utils.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavformat/utils.c b/libavformat/utils.c index ff5e14df6c..65d111459f 100644 --- a/libavformat/utils.c +++ b/libavformat/utils.c @@ -1738,8 +1738,8 @@ int av_read_frame(AVFormatContext *s, AVPacket *pkt) // current one had no dts, we will set this to AV_NOPTS_VALUE. int64_t last_dts = next_pkt->dts; while (pktl && next_pkt->pts == AV_NOPTS_VALUE) { - if (pktl->pkt.stream_index == next_pkt->stream_index && - (av_compare_mod(next_pkt->dts, pktl->pkt.dts, 2LL << (wrap_bits - 1)) < 0)) { + if (pktl->pkt.stream_index == next_pkt->stream_index && wrap_bits < 63 && + av_compare_mod(next_pkt->dts, pktl->pkt.dts, 2LL << (wrap_bits - 1)) < 0) { if (av_compare_mod(pktl->pkt.pts, pktl->pkt.dts, 2LL << (wrap_bits - 1))) { // not B-frame next_pkt->pts = pktl->pkt.dts; -- 2.15.0.448.gf294e3d99a-goog
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel