Andreas Rheinhardt: > This check is intended to be avoid buffer overflows, > yet there are four problems with it: > 1. It has an in-built off-by-one error: len == out_end - out > is perfectly fine and nothing to worry about. > This off-by-one error led to the pixel in the lower-right corner > not being set properly for the back frame of the sample from > the rl2 FATE-test. This pixel is copied to every frame which > is the reason for the update to the reference file of said test. > With this patch, the output of the decoder matches the output > as captured from the reference decoder* (apart from the fact > that said reference somehow lacks the top part of the frame > (copied over from the background frame)). > 2. Given that the stride of the buffer may be different > from the width of the video (despite one pixel taking one byte), > there is a second check lateron making the first check redundant > (if one returns immediately; a simple break at the second check > is not sufficient, because it only exits the inner loop). > 3. The check is based around the assumption of the stride being > positive (it has this in common with the other check which > will be fixed in a future commit). > 4. Even after fixing the off-by-one error, the check in > question is still triggered by all the non-background frames > in the FATE sample as well as by A1100100.RL2. In all these > cases, they use len == 255 and val == 128. For videos with > background frame this just means "copy from the background > frame", which would be done anyway lateron.* Yet for videos > without it copying it is necessary to avoid leaving > uninitialized parts in the video. > > *: Available in https://samples.mplayerhq.hu/game-formats/voyeur-rl2/ > **: Due to this, the code that copies the rest from the > back frame is no longer executed for any of the samples > available on the sample server. Given that these are only > the files from the demo version of this game, I don't know > whether this code is executed for any file in existence or not. > > Signed-off-by: Andreas Rheinhardt <andreas.rheinha...@outlook.com> > --- > libavcodec/rl2.c | 5 +- > tests/ref/fate/rl2 | 216 ++++++++++++++++++++++----------------------- > 2 files changed, 109 insertions(+), 112 deletions(-) > > diff --git a/libavcodec/rl2.c b/libavcodec/rl2.c > index 5dedb96266..2464ad59ac 100644 > --- a/libavcodec/rl2.c > +++ b/libavcodec/rl2.c > @@ -91,9 +91,6 @@ static void rl2_rle_decode(Rl2Context *s, const uint8_t > *in, int size, > break; > } > > - if (len >= out_end - out) > - break; > - > if (s->back_frame) > val |= 0x80; > else > @@ -106,7 +103,7 @@ static void rl2_rle_decode(Rl2Context *s, const uint8_t > *in, int size, > out += stride_adj; > line_end += stride; > if (len >= out_end - out) > - break; > + return; > } > } > } > diff --git a/tests/ref/fate/rl2 b/tests/ref/fate/rl2 > index 9189822503..a3c6a75d05 100644 > --- a/tests/ref/fate/rl2 > +++ b/tests/ref/fate/rl2 > @@ -3,111 +3,111 @@ > #codec_id 0: rawvideo > #dimensions 0: 320x200 > #sar 0: 0/1 > -0, 0, 0, 1, 192000, 0x7112a667 > -0, 1, 1, 1, 192000, 0x6936abf3 > -0, 2, 2, 1, 192000, 0xb1f08981 > -0, 3, 3, 1, 192000, 0x4ce7fece > -0, 4, 4, 1, 192000, 0xf04decde > -0, 5, 5, 1, 192000, 0x47fef74b > -0, 6, 6, 1, 192000, 0x99b42ac2 > -0, 7, 7, 1, 192000, 0x3c4c419d > -0, 8, 8, 1, 192000, 0x66bf5588 > -0, 9, 9, 1, 192000, 0xe1de4725 > -0, 10, 10, 1, 192000, 0x348b2af9 > -0, 11, 11, 1, 192000, 0x1ce73e83 > -0, 12, 12, 1, 192000, 0xcdaa6e02 > -0, 13, 13, 1, 192000, 0x370dc2ce > -0, 14, 14, 1, 192000, 0x1e1e40fe > -0, 15, 15, 1, 192000, 0x491470a4 > -0, 16, 16, 1, 192000, 0x88c43e9a > -0, 17, 17, 1, 192000, 0x036f3f44 > -0, 18, 18, 1, 192000, 0xc8be7e25 > -0, 19, 19, 1, 192000, 0xbedb397d > -0, 20, 20, 1, 192000, 0x97c410f4 > -0, 21, 21, 1, 192000, 0xc2c8225d > -0, 22, 22, 1, 192000, 0xe396bccb > -0, 23, 23, 1, 192000, 0x7e89c24c > -0, 24, 24, 1, 192000, 0xb044954c > -0, 25, 25, 1, 192000, 0x335890dd > -0, 26, 26, 1, 192000, 0x58a457c0 > -0, 27, 27, 1, 192000, 0xeb0f4798 > -0, 28, 28, 1, 192000, 0x0bfc39a1 > -0, 29, 29, 1, 192000, 0x06a6905a > -0, 30, 30, 1, 192000, 0x5300c99b > -0, 31, 31, 1, 192000, 0x38f3f845 > -0, 32, 32, 1, 192000, 0x6afa3543 > -0, 33, 33, 1, 192000, 0x5106a8e0 > -0, 34, 34, 1, 192000, 0xc76f0dab > -0, 35, 35, 1, 192000, 0x8efa6939 > -0, 36, 36, 1, 192000, 0x64ea23d7 > -0, 37, 37, 1, 192000, 0x421a2817 > -0, 38, 38, 1, 192000, 0xc854fa18 > -0, 39, 39, 1, 192000, 0x4a10d59c > -0, 40, 40, 1, 192000, 0x72637829 > -0, 41, 41, 1, 192000, 0xdbbe2796 > -0, 42, 42, 1, 192000, 0xff742e6b > -0, 43, 43, 1, 192000, 0xf94b9346 > -0, 44, 44, 1, 192000, 0xc90ea53c > -0, 45, 45, 1, 192000, 0x177483bb > -0, 46, 46, 1, 192000, 0x3510369c > -0, 47, 47, 1, 192000, 0x501034bf > -0, 48, 48, 1, 192000, 0x315c744b > -0, 49, 49, 1, 192000, 0xdb5048ae > -0, 50, 50, 1, 192000, 0x09a86221 > -0, 51, 51, 1, 192000, 0xb9c9568a > -0, 52, 52, 1, 192000, 0x5eee665b > -0, 53, 53, 1, 192000, 0xdef85517 > -0, 54, 54, 1, 192000, 0x7896b5ad > -0, 55, 55, 1, 192000, 0x19fbad39 > -0, 56, 56, 1, 192000, 0x63358748 > -0, 57, 57, 1, 192000, 0xca0196a3 > -0, 58, 58, 1, 192000, 0x0b321da3 > -0, 59, 59, 1, 192000, 0xa07af07e > -0, 60, 60, 1, 192000, 0x21f9310c > -0, 61, 61, 1, 192000, 0x62d59874 > -0, 62, 62, 1, 192000, 0x021227b4 > -0, 63, 63, 1, 192000, 0x01dac0c2 > -0, 64, 64, 1, 192000, 0xf3c33a74 > -0, 65, 65, 1, 192000, 0xeebe83b7 > -0, 66, 66, 1, 192000, 0x9ec77f97 > -0, 67, 67, 1, 192000, 0xc91c2e37 > -0, 68, 68, 1, 192000, 0x7b58751d > -0, 69, 69, 1, 192000, 0xb178dfbb > -0, 70, 70, 1, 192000, 0x2a63b5be > -0, 71, 71, 1, 192000, 0x44a407ac > -0, 72, 72, 1, 192000, 0x9a8e17dd > -0, 73, 73, 1, 192000, 0x5546f4c8 > -0, 74, 74, 1, 192000, 0xeda94586 > -0, 75, 75, 1, 192000, 0x941dfa6b > -0, 76, 76, 1, 192000, 0x9772301d > -0, 77, 77, 1, 192000, 0x8be16b16 > -0, 78, 78, 1, 192000, 0x26dd4496 > -0, 79, 79, 1, 192000, 0x27823797 > -0, 80, 80, 1, 192000, 0xacc914f7 > -0, 81, 81, 1, 192000, 0xcbb72c9b > -0, 82, 82, 1, 192000, 0x4bd3391c > -0, 83, 83, 1, 192000, 0x4e6adbfe > -0, 84, 84, 1, 192000, 0x7a791c75 > -0, 85, 85, 1, 192000, 0xc4f59c94 > -0, 86, 86, 1, 192000, 0xc4f59c94 > -0, 87, 87, 1, 192000, 0x984a4a0b > -0, 88, 88, 1, 192000, 0x3353f31f > -0, 89, 89, 1, 192000, 0xa9d4dc5a > -0, 90, 90, 1, 192000, 0xb33425d0 > -0, 91, 91, 1, 192000, 0x546d768a > -0, 92, 92, 1, 192000, 0xfefbe5c9 > -0, 93, 93, 1, 192000, 0xbd6be61d > -0, 94, 94, 1, 192000, 0xf5792731 > -0, 95, 95, 1, 192000, 0xccde0582 > -0, 96, 96, 1, 192000, 0x857d58ee > -0, 97, 97, 1, 192000, 0xe914ce48 > -0, 98, 98, 1, 192000, 0x1f736298 > -0, 99, 99, 1, 192000, 0xec0b4846 > -0, 100, 100, 1, 192000, 0xe1422624 > -0, 101, 101, 1, 192000, 0x56e2b0e0 > -0, 102, 102, 1, 192000, 0xc4190640 > -0, 103, 103, 1, 192000, 0x7c461977 > -0, 104, 104, 1, 192000, 0x34b1d5e8 > -0, 105, 105, 1, 192000, 0xbdc70f7a > -0, 106, 106, 1, 192000, 0xb466cd8d > -0, 107, 107, 1, 192000, 0x0e86a04c > +0, 0, 0, 1, 192000, 0x7324a772 > +0, 1, 1, 1, 192000, 0x6b48acfe > +0, 2, 2, 1, 192000, 0xb4028a8c > +0, 3, 3, 1, 192000, 0x4ef9ffd9 > +0, 4, 4, 1, 192000, 0xf25fede9 > +0, 5, 5, 1, 192000, 0x4a10f856 > +0, 6, 6, 1, 192000, 0x9bc62bcd > +0, 7, 7, 1, 192000, 0x3e5e42a8 > +0, 8, 8, 1, 192000, 0x68d15693 > +0, 9, 9, 1, 192000, 0xe3f04830 > +0, 10, 10, 1, 192000, 0x369d2c04 > +0, 11, 11, 1, 192000, 0x1ef93f8e > +0, 12, 12, 1, 192000, 0xcfbc6f0d > +0, 13, 13, 1, 192000, 0x391fc3d9 > +0, 14, 14, 1, 192000, 0x20304209 > +0, 15, 15, 1, 192000, 0x4b2671af > +0, 16, 16, 1, 192000, 0x8ad63fa5 > +0, 17, 17, 1, 192000, 0x0581404f > +0, 18, 18, 1, 192000, 0xcad07f30 > +0, 19, 19, 1, 192000, 0xc0ed3a88 > +0, 20, 20, 1, 192000, 0x99d611ff > +0, 21, 21, 1, 192000, 0xc4da2368 > +0, 22, 22, 1, 192000, 0xe5a8bdd6 > +0, 23, 23, 1, 192000, 0x809bc357 > +0, 24, 24, 1, 192000, 0xb2569657 > +0, 25, 25, 1, 192000, 0x356a91e8 > +0, 26, 26, 1, 192000, 0x5ab658cb > +0, 27, 27, 1, 192000, 0xed2148a3 > +0, 28, 28, 1, 192000, 0x0e0e3aac > +0, 29, 29, 1, 192000, 0x08b89165 > +0, 30, 30, 1, 192000, 0x5512caa6 > +0, 31, 31, 1, 192000, 0x3b05f950 > +0, 32, 32, 1, 192000, 0x6d0c364e > +0, 33, 33, 1, 192000, 0x5318a9eb > +0, 34, 34, 1, 192000, 0xc9810eb6 > +0, 35, 35, 1, 192000, 0x910c6a44 > +0, 36, 36, 1, 192000, 0x66fc24e2 > +0, 37, 37, 1, 192000, 0x442c2922 > +0, 38, 38, 1, 192000, 0xca66fb23 > +0, 39, 39, 1, 192000, 0x4c22d6a7 > +0, 40, 40, 1, 192000, 0x74757934 > +0, 41, 41, 1, 192000, 0xddd028a1 > +0, 42, 42, 1, 192000, 0x01952f76 > +0, 43, 43, 1, 192000, 0xfb5d9451 > +0, 44, 44, 1, 192000, 0xcb20a647 > +0, 45, 45, 1, 192000, 0x198684c6 > +0, 46, 46, 1, 192000, 0x372237a7 > +0, 47, 47, 1, 192000, 0x522235ca > +0, 48, 48, 1, 192000, 0x336e7556 > +0, 49, 49, 1, 192000, 0xdd6249b9 > +0, 50, 50, 1, 192000, 0x0bba632c > +0, 51, 51, 1, 192000, 0xbbdb5795 > +0, 52, 52, 1, 192000, 0x61006766 > +0, 53, 53, 1, 192000, 0xe10a5622 > +0, 54, 54, 1, 192000, 0x7aa8b6b8 > +0, 55, 55, 1, 192000, 0x1c0dae44 > +0, 56, 56, 1, 192000, 0x65478853 > +0, 57, 57, 1, 192000, 0xcc1397ae > +0, 58, 58, 1, 192000, 0x0d441eae > +0, 59, 59, 1, 192000, 0xa28cf189 > +0, 60, 60, 1, 192000, 0x240b3217 > +0, 61, 61, 1, 192000, 0x64e7997f > +0, 62, 62, 1, 192000, 0x042428bf > +0, 63, 63, 1, 192000, 0x03ecc1cd > +0, 64, 64, 1, 192000, 0xf5d53b7f > +0, 65, 65, 1, 192000, 0xf0d084c2 > +0, 66, 66, 1, 192000, 0xa0d980a2 > +0, 67, 67, 1, 192000, 0xcb2e2f42 > +0, 68, 68, 1, 192000, 0x7d6a7628 > +0, 69, 69, 1, 192000, 0xb38ae0c6 > +0, 70, 70, 1, 192000, 0x2c75b6c9 > +0, 71, 71, 1, 192000, 0x46b608b7 > +0, 72, 72, 1, 192000, 0x9ca018e8 > +0, 73, 73, 1, 192000, 0x5758f5d3 > +0, 74, 74, 1, 192000, 0xefbb4691 > +0, 75, 75, 1, 192000, 0x962ffb76 > +0, 76, 76, 1, 192000, 0x99843128 > +0, 77, 77, 1, 192000, 0x8df36c21 > +0, 78, 78, 1, 192000, 0x28ef45a1 > +0, 79, 79, 1, 192000, 0x299438a2 > +0, 80, 80, 1, 192000, 0xaedb1602 > +0, 81, 81, 1, 192000, 0xcdc92da6 > +0, 82, 82, 1, 192000, 0x4de53a27 > +0, 83, 83, 1, 192000, 0x507cdd09 > +0, 84, 84, 1, 192000, 0x7c8b1d80 > +0, 85, 85, 1, 192000, 0xc7079d9f > +0, 86, 86, 1, 192000, 0xc7079d9f > +0, 87, 87, 1, 192000, 0x9a5c4b16 > +0, 88, 88, 1, 192000, 0x3565f42a > +0, 89, 89, 1, 192000, 0xabe6dd65 > +0, 90, 90, 1, 192000, 0xb54626db > +0, 91, 91, 1, 192000, 0x567f7795 > +0, 92, 92, 1, 192000, 0x011ce6d4 > +0, 93, 93, 1, 192000, 0xbf7de728 > +0, 94, 94, 1, 192000, 0xf78b283c > +0, 95, 95, 1, 192000, 0xcef0068d > +0, 96, 96, 1, 192000, 0x878f59f9 > +0, 97, 97, 1, 192000, 0xeb26cf53 > +0, 98, 98, 1, 192000, 0x218563a3 > +0, 99, 99, 1, 192000, 0xee1d4951 > +0, 100, 100, 1, 192000, 0xe354272f > +0, 101, 101, 1, 192000, 0x58f4b1eb > +0, 102, 102, 1, 192000, 0xc62b074b > +0, 103, 103, 1, 192000, 0x7e581a82 > +0, 104, 104, 1, 192000, 0x36c3d6f3 > +0, 105, 105, 1, 192000, 0xbfd91085 > +0, 106, 106, 1, 192000, 0xb678ce98 > +0, 107, 107, 1, 192000, 0x1098a157
Will apply this patchset tonight unless there are objections. - Andreas _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
