> From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On Behalf > Of Pengfei Qu > Sent: Monday, March 26, 2018 1:51 PM > To: ffmpeg-devel@ffmpeg.org > Cc: Qu, Pengfei <pengfei...@intel.com> > Subject: [FFmpeg-devel] [V2 1/2] lavc/vaapi_encode: fix the caculation > overflow > > this fix the overflow during the caculation before value assignment. > > Signed-off-by: Pengfei Qu <pengfei...@intel.com> > --- > libavcodec/vaapi_encode.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index > 36c85a3..78347d4 100644 > --- a/libavcodec/vaapi_encode.c > +++ b/libavcodec/vaapi_encode.c > @@ -1168,9 +1168,9 @@ static av_cold int > vaapi_encode_init_rate_control(AVCodecContext *avctx) > rc_target_percentage = 100; > } else { > rc_bits_per_second = avctx->rc_max_rate; > - rc_target_percentage = (avctx->bit_rate * 100) / > rc_bits_per_second; > + rc_target_percentage = (unsigned long)(avctx->bit_rate * > + 100) / rc_bits_per_second; > } > - rc_window_size = (hrd_buffer_size * 1000) / avctx->bit_rate; > + rc_window_size = (unsigned long)(hrd_buffer_size * 1000) / > + avctx->bit_rate;
Should better to use " uint64_t". > } > > ctx->rc_params.misc.type = VAEncMiscParameterTypeRateControl; > -- > 2.9.3 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
