On 12/03/18 05:38, Pengfei Qu wrote:
> this fix the overflow during the caculation before value assignment.
>
> Signed-off-by: Pengfei Qu <pengfei...@intel.com>
> ---
> libavcodec/vaapi_encode.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
> index 36c85a3..78347d4 100644
> --- a/libavcodec/vaapi_encode.c
> +++ b/libavcodec/vaapi_encode.c
> @@ -1168,9 +1168,9 @@ static av_cold int
> vaapi_encode_init_rate_control(AVCodecContext *avctx)
> rc_target_percentage = 100;
> } else {
> rc_bits_per_second = avctx->rc_max_rate;
> - rc_target_percentage = (avctx->bit_rate * 100) /
> rc_bits_per_second;
> + rc_target_percentage = (unsigned long)(avctx->bit_rate * 100) /
> rc_bits_per_second;
What is the problem here? All this will do is truncate the 64-bit value on
platforms with 32-bit long.
> }
> - rc_window_size = (hrd_buffer_size * 1000) / avctx->bit_rate;
> + rc_window_size = (unsigned long)(hrd_buffer_size * 1000) /
> avctx->bit_rate;
Perhaps you want to ensure that the multiplication doesn't overflow rather than
truncating the result after it has overflowed.
> }
>
> ctx->rc_params.misc.type = VAEncMiscParameterTypeRateControl;
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
