On 13.11.2015 02:08, Vittorio Giovara wrote:
> oh I see, that can happen for a special crafted file, DDPF_FOURCC has
> been introduced recently while DDPF_PALETTE has been removed, so a
> normal file should not have both set.
OK, that makes sense.
> Because of that, and how rare palette dds are, I think it's quite safe
> to unset ctx->paletted if ctx->compressed is set, and be done with it.
Patch doing that is attached.
> That is the only case I can see that happen, right?
Yes, that's the only case.
> Thanks for the catch, how did you find it btw? :)
Like all the other: with afl [1].
It's really good at finding weird cases. ;)
Best regards,
Andreas
1: http://lcamtuf.coredump.cx/afl/
>From 6e55b4ffffc60e93168236c0f05e67e89f0007da Mon Sep 17 00:00:00 2001
From: Andreas Cadhalpun <andreas.cadhal...@googlemail.com>
Date: Fri, 13 Nov 2015 21:48:27 +0100
Subject: [PATCH] dds: disable palette flag for compressed dds
Having both is not valid and can cause a NULL pointer dereference of
frame->data[1] later.
Signed-off-by: Andreas Cadhalpun <andreas.cadhal...@googlemail.com>
---
libavcodec/dds.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/libavcodec/dds.c b/libavcodec/dds.c
index f04a4f5..d473fd1 100644
--- a/libavcodec/dds.c
+++ b/libavcodec/dds.c
@@ -141,6 +141,12 @@ static int parse_pixel_format(AVCodecContext *avctx)
normal_map = flags & DDPF_NORMALMAP;
fourcc = bytestream2_get_le32(gbc);
+ if (ctx->compressed && ctx->paletted) {
+ av_log(avctx, AV_LOG_WARNING,
+ "Disabling invalid palette flag for compressed dds.\n");
+ ctx->paletted = 0;
+ }
+
bpp = bytestream2_get_le32(gbc); // rgbbitcount
r = bytestream2_get_le32(gbc); // rbitmask
g = bytestream2_get_le32(gbc); // gbitmask
--
2.6.2
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
