On Tue, 2021-07-06 at 18:19 +0200, Henrik Gramner wrote:
> On Mon, Jul 5, 2021 at 4:32 AM Fei Wang wrote:
> > +int64_t v, w;
> > +int32_t *param = &s->cur_frame.gm_params[idx][0];
>
> ...
> > +v = param[4] * (1 << AV1_WARPEDMODEL_PREC_BITS);
> > +w = param[3] * param[4];
>
On Mon, Jul 5, 2021 at 4:32 AM Fei Wang wrote:
> +int64_t v, w;
> +int32_t *param = &s->cur_frame.gm_params[idx][0];
...
> +v = param[4] * (1 << AV1_WARPEDMODEL_PREC_BITS);
> +w = param[3] * param[4];
Possible integer overflow? Might need some int64_t casting before the
mu
On Mon, 2021-07-05 at 07:12 +0200, Jean-Baptiste Kempf wrote:
> On Mon, 5 Jul 2021, at 04:29, Fei Wang wrote:
> > Defined in spec 7.11.3.6/7.11.3.7.
>
> ...
> > +static const uint16_t div_lut[AV1_DIV_LUT_NUM] = {
> > + 16384, 16320, 16257, 16194, 16132, 16070, 16009, 15948, 15888,
> > 15828, 1
On Mon, 5 Jul 2021, at 04:29, Fei Wang wrote:
> Defined in spec 7.11.3.6/7.11.3.7.
...
> +static const uint16_t div_lut[AV1_DIV_LUT_NUM] = {
> + 16384, 16320, 16257, 16194, 16132, 16070, 16009, 15948, 15888,
> 15828, 15768,
Where are those numbers coming from? From the spec? From an annex?
The
Defined in spec 7.11.3.6/7.11.3.7.
Signed-off-by: Fei Wang
---
libavcodec/av1dec.c | 97 +
libavcodec/av1dec.h | 1 +
2 files changed, 98 insertions(+)
diff --git a/libavcodec/av1dec.c b/libavcodec/av1dec.c
index 1dda0f9160..3fca17e84b 100644
--- a/l
