Hi,
encoding large frames with rv10 currently results in an assertion, because the
number of macro blocks doesn't fit into 12 bits.
Attached patch avoids this by just not trying to write the macro block number in
that case. The decoder can figure it out on it's own:
/* if multiple packets per frame are sent, the position at which
* to display the macroblocks is coded here */
mb_xy = s->mb_x + s->mb_y * s->mb_width;
if (show_bits(&s->gb, 12) == 0 || (mb_xy && mb_xy < s->mb_num)) {
s->mb_x = get_bits(&s->gb, 6); /* mb_x */
s->mb_y = get_bits(&s->gb, 6); /* mb_y */
mb_count = get_bits(&s->gb, 12);
} else {
s->mb_x = 0;
s->mb_y = 0;
mb_count = s->mb_width * s->mb_height;
}
The other option would be to abort encoding with an error, but
ff_rv20_encode_picture_header currently returns void.
Best regards,
Andreas
>From 4f161ce1c709d606ba33e511303e3a88aa70f298 Mon Sep 17 00:00:00 2001
From: Andreas Cadhalpun <andreas.cadhal...@googlemail.com>
Date: Mon, 2 Mar 2015 17:22:25 +0100
Subject: [PATCH] avcodec/rv10: check size of s->mb_width * s->mb_height
If it doesn't fit into 12 bits, don't try to put it there, as it
triggers an assertion.
The information is not necessary anyway, because the decoder calculates
it if it's not in the bitstream.
Signed-off-by: Andreas Cadhalpun <andreas.cadhal...@googlemail.com>
---
libavcodec/rv10enc.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/libavcodec/rv10enc.c b/libavcodec/rv10enc.c
index 2541132..00addda 100644
--- a/libavcodec/rv10enc.c
+++ b/libavcodec/rv10enc.c
@@ -47,7 +47,7 @@ void ff_rv10_encode_picture_header(MpegEncContext *s, int picture_number)
}
/* if multiple packets per frame are sent, the position at which
to display the macroblocks is coded here */
- if(!full_frame){
+ if(!full_frame && s->mb_width * s->mb_height < (1U << 12)){
put_bits(&s->pb, 6, 0); /* mb_x */
put_bits(&s->pb, 6, 0); /* mb_y */
put_bits(&s->pb, 12, s->mb_width * s->mb_height);
--
2.1.4
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
