Re: [FFmpeg-devel] [libav-devel] [PATCH] vp8: check for too large dimensions

Hi Ronald, On 08.06.2015 23:18, Ronald S. Bultje wrote: > On Mon, Jun 8, 2015 at 5:08 PM, Andreas Cadhalpun < > andreas.cadhal...@googlemail.com> wrote: > >> On 07.06.2015 22:39, Michael Niedermayer wrote: >>> On Sun, Jun 07, 2015 at 04:27:42PM -0400, Ronald S. Bultje wrote: So what happens

Re: [FFmpeg-devel] [libav-devel] [PATCH] vp8: check for too large dimensions

Hi, On Mon, Jun 8, 2015 at 5:08 PM, Andreas Cadhalpun < andreas.cadhal...@googlemail.com> wrote: > On 07.06.2015 22:39, Michael Niedermayer wrote: > > On Sun, Jun 07, 2015 at 04:27:42PM -0400, Ronald S. Bultje wrote: > >> So what happens when you change mv_max/min to be integers (instead of > >>

Re: [FFmpeg-devel] [libav-devel] [PATCH] vp8: check for too large dimensions

On 07.06.2015 22:39, Michael Niedermayer wrote: > On Sun, Jun 07, 2015 at 04:27:42PM -0400, Ronald S. Bultje wrote: >> So what happens when you change mv_max/min to be integers (instead of >> int16_t) without further touching VP56mv, e.g. by making mv_max/min a >> VP8intminmaxmv (a newly created ty

Re: [FFmpeg-devel] [libav-devel] [PATCH] vp8: check for too large dimensions

Hi, On Sun, Jun 7, 2015 at 4:39 PM, Michael Niedermayer wrote: > On Sun, Jun 07, 2015 at 04:27:42PM -0400, Ronald S. Bultje wrote: > > Hi, > > > > On Sun, Jun 7, 2015 at 2:35 PM, Andreas Cadhalpun < > > andreas.cadhal...@googlemail.com> wrote: > > > > > Hi Ronald, > > > > > > On 07.06.2015 19:37

Re: [FFmpeg-devel] [libav-devel] [PATCH] vp8: check for too large dimensions

On Sun, Jun 07, 2015 at 04:27:42PM -0400, Ronald S. Bultje wrote: > Hi, > > On Sun, Jun 7, 2015 at 2:35 PM, Andreas Cadhalpun < > andreas.cadhal...@googlemail.com> wrote: > > > Hi Ronald, > > > > On 07.06.2015 19:37, Ronald S. Bultje wrote: > > > On Sun, Jun 7, 2015 at 12:44 PM, Andreas Cadhalpun

Re: [FFmpeg-devel] [libav-devel] [PATCH] vp8: check for too large dimensions

Hi, On Sun, Jun 7, 2015 at 2:35 PM, Andreas Cadhalpun < andreas.cadhal...@googlemail.com> wrote: > Hi Ronald, > > On 07.06.2015 19:37, Ronald S. Bultje wrote: > > On Sun, Jun 7, 2015 at 12:44 PM, Andreas Cadhalpun < > > andreas.cadhal...@googlemail.com> wrote: > >> On 07.06.2015 17:52, Ronald S.

Re: [FFmpeg-devel] [libav-devel] [PATCH] vp8: check for too large dimensions

Hi Ronald, On 07.06.2015 19:37, Ronald S. Bultje wrote: > On Sun, Jun 7, 2015 at 12:44 PM, Andreas Cadhalpun < > andreas.cadhal...@googlemail.com> wrote: >> On 07.06.2015 17:52, Ronald S. Bultje wrote: >>> We can't simply claim that 8k is not a valid dimension, that would make us >>> a vp8-incompa

Re: [FFmpeg-devel] [libav-devel] [PATCH] vp8: check for too large dimensions

Hi, On Sun, Jun 7, 2015 at 12:44 PM, Andreas Cadhalpun < andreas.cadhal...@googlemail.com> wrote: > Hi Ronald, > > On 07.06.2015 17:52, Ronald S. Bultje wrote: > > On Sun, Jun 7, 2015 at 10:05 AM, Andreas Cadhalpun < > > andreas.cadhal...@googlemail.com> wrote: > > > >> +#define MARGIN (16 << 2)

Re: [FFmpeg-devel] [libav-devel] [PATCH] vp8: check for too large dimensions

Hi Ronald, On 07.06.2015 17:52, Ronald S. Bultje wrote: > On Sun, Jun 7, 2015 at 10:05 AM, Andreas Cadhalpun < > andreas.cadhal...@googlemail.com> wrote: > >> +#define MARGIN (16 << 2) >> +#define MAX_MB_SIZE (((INT16_MAX - MARGIN) >> 6) + 1) >> > > So this is roughly 9 bits. > > >> +if (s

Re: [FFmpeg-devel] [libav-devel] [PATCH] vp8: check for too large dimensions

Hi, On Sun, Jun 7, 2015 at 10:05 AM, Andreas Cadhalpun < andreas.cadhal...@googlemail.com> wrote: > +#define MARGIN (16 << 2) > +#define MAX_MB_SIZE (((INT16_MAX - MARGIN) >> 6) + 1) > So this is roughly 9 bits. > +if (s->avctx->coded_width > MAX_MB_SIZE * 16 || > +s->avctx->coded