The overflow guard for the ambisonic channel count itself evaluated order + 1 in int arithmetic, which is undefined when order == INT_MAX (a value reachable from strtol on user-supplied input such as 'ambisonic 2147483647'). On overflow the check was bypassed and the function returned success with a bogus nb_channels of 0. Perform the (order + 1)^2 range check in int64_t so no intermediate int overflow can occur.
Signed-off-by: Anas Khan <[email protected]> --- libavutil/channel_layout.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavutil/channel_layout.c b/libavutil/channel_layout.c index 4f18047df3..e26b7b4949 100644 --- a/libavutil/channel_layout.c +++ b/libavutil/channel_layout.c @@ -338,7 +338,7 @@ int av_channel_layout_from_string(AVChannelLayout *channel_layout, int order; order = strtol(p, &endptr, 0); - if (order < 0 || order + 1 > INT_MAX / (order + 1) || + if (order < 0 || ((int64_t)order + 1) * ((int64_t)order + 1) > INT_MAX || (*endptr && *endptr != '+')) return AVERROR(EINVAL); -- 2.54.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
