On 24 June 2016 at 02:36, Michael Niedermayer <mich...@niedermayer.cc>
wrote:
> On Thu, Jun 23, 2016 at 06:06:59PM +0100, Rostislav Pehlivanov wrote:
> > Prevents having to have random magic values in the decoder and a
> > separate macro in the encoder.
> >
> > Signed-off-by: Rostislav Pehlivanov <rpehliva...@obe.tv>
> > ---
> > libavcodec/diracdec.c | 8 ++++----
> > libavcodec/diractab.h | 2 ++
> > libavcodec/vc2enc.c | 9 +++------
> > 3 files changed, 9 insertions(+), 10 deletions(-)
> >
> > diff --git a/libavcodec/diracdec.c b/libavcodec/diracdec.c
> > index c8ab2df..48ba194 100644
> > --- a/libavcodec/diracdec.c
> > +++ b/libavcodec/diracdec.c
> > @@ -486,7 +486,7 @@ static inline void codeblock(DiracContext *s,
> SubBand *b,
> > b->quant = quant;
> > }
> >
> > - if (b->quant > 115) {
> > + if (b->quant > DIRAC_MAX_QUANT_INDEX) {
> > av_log(s->avctx, AV_LOG_ERROR, "Unsupported quant %d\n",
> b->quant);
> > b->quant = 0;
> > return;
> > @@ -676,12 +676,12 @@ static void decode_subband(DiracContext *s,
> GetBitContext *gb, int quant,
> > uint8_t *buf2 = b2 ? b2->ibuf + top * b2->stride: NULL;
> > int x, y;
> >
> > - if (quant > 115) {
> > + if (quant > DIRAC_MAX_QUANT_INDEX) {
> > av_log(s->avctx, AV_LOG_ERROR, "Unsupported quant %d\n", quant);
> > return;
> > }
>
> > - qfactor = ff_dirac_qscale_tab[quant & 0x7f];
> > - qoffset = ff_dirac_qoffset_intra_tab[quant & 0x7f] + 2;
> > + qfactor = ff_dirac_qscale_tab[quant & DIRAC_MAX_QUANT_INDEX];
> > + qoffset = ff_dirac_qoffset_intra_tab[quant & DIRAC_MAX_QUANT_INDEX]
> + 2;
>
> if iam not missing anything then
> DIRAC_MAX_QUANT_INDEX / FF_ARRAY_ELEMS(ff_dirac_qscale_tab) is
> 116
> quant & 116 looks unintended
>
>
> [...]
> --
> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> I know you won't believe me, but the highest form of Human Excellence is
> to question oneself and others. -- Socrates
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>
That AND was originally protection for invalid quantization indices causing
an out of bounds access.
Forgot to remove them, changed locally.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
