[PATCH] vb2: check memory model for VIDIOC_CREATE_BUFS

[Hans Verkuil]

vb2_core_create_bufs did not check if the memory model for newly added
buffers is the same as for already existing buffers. It should return an
error if they aren't the same.

Signed-off-by: Hans Verkuil <hans.verk...@cisco.com>
Reported-by: syzbot+e1fb118a2ebb88031...@syzkaller.appspotmail.com
Cc: <sta...@vger.kernel.org>      # for v4.16 and up
---
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
b/drivers/media/common/videobuf2/videobuf2-core.c
index 975ff5669f72..c49c67473408 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -812,6 +812,9 @@ int vb2_core_create_bufs(struct vb2_queue *q, enum 
vb2_memory memory,
                memset(q->alloc_devs, 0, sizeof(q->alloc_devs));
                q->memory = memory;
                q->waiting_for_buffers = !q->is_output;
+       } else if (q->memory != memory) {
+               dprintk(1, "memory model mismatch\n");
+               return -EINVAL;
        }

        num_buffers = min(*count, VB2_MAX_FRAME - q->num_buffers);