On Fri, Dec 4, 2015 at 2:23 PM, Andreas Cadhalpun
<andreas.cadhal...@googlemail.com> wrote:
> If minq is negative, the range of sf_idx can be larger than
> SCALE_MAX_DIFF allows, causing assertion failures later in
> encode_scale_factors.
>
> Signed-off-by: Andreas Cadhalpun <andreas.cadhal...@googlemail.com>
> ---
> libavcodec/aaccoder.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/libavcodec/aaccoder.c b/libavcodec/aaccoder.c
> index 2a0cb1f..e8a61ce 100644
> --- a/libavcodec/aaccoder.c
> +++ b/libavcodec/aaccoder.c
> @@ -370,7 +370,7 @@ static void search_for_quantizers_anmr(AVCodecContext
> *avctx, AACEncContext *s,
> }
> while (idx) {
> sce->sf_idx[bandaddr[idx]] = minq + q0;
> - minq = paths[idx][minq].prev;
> + minq = FFMAX(paths[idx][minq].prev, 0);
> idx--;
> }
> //set the same quantizers inside window groups
Actually, a negative .prev signals an impossible path.
So perhaps the attached patch (well, git diff) would go better? (if
you confirm it passes fuzzing I'll push it)
diff --git a/libavcodec/aaccoder.c b/libavcodec/aaccoder.c
index 2a0cb1f..b3c8eea 100644
--- a/libavcodec/aaccoder.c
+++ b/libavcodec/aaccoder.c
@@ -363,7 +363,7 @@ static void search_for_quantizers_anmr(AVCodecContext *avctx, AACEncContext *s,
mincost = paths[idx][0].cost;
minq = 0;
for (i = 1; i < TRELLIS_STATES; i++) {
- if (paths[idx][i].cost < mincost) {
+ if (paths[idx][i].cost < mincost && paths[idx][i].prev >= 0) {
mincost = paths[idx][i].cost;
minq = i;
}
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
