On 07/20/2015 06:07 AM, Peter Maydell wrote: > On 16 July 2015 at 19:38, Wei Huang <w...@redhat.com> wrote: >> Recently we found that virtio-console devices consumes lots AArch64 guest >> memory, roughly 1GB with 8 devices. After debugging, it turns out that lots >> of factors contribute to this problem: i) guest PAGE_SIZE=64KB, ii) >> virtio-mmio based devices, and iii) virtio-console device. Here is the >> detailed analysis: >> >> 1. First, during initialization, virtio-mmio driver in guest pokes vq >> size by reading VIRTIO_MMIO_QUEUE_NUM_MAX (see virtio_mmio.c file). >> 2. QEMU returns VIRTQUEUE_MAX_SIZE (1024) to guest VM; And virtio-mmio uses >> it as the default vq size. >> 3. virtio-console driver allocates vring buffers based on this value (see >> add_inbuf() function of virtio_console.c file). Because PAGE_SIZE=64KB, >> ~64MB is allocated for each virtio-console vq. >> >> This patch addresses the problem by returning the iniatlized vring size >> when VM queries QEMU about VIRTIO_MMIO_QUEUE_NUM_MAX. This is similar to >> virtio-pci's approach. By doing this, the vq memory consumption is reduced >> substantially. > > I don't know if this patch is sensible to apply anyway, but from > this description this really sounds like a guest kernel bug. > QEMU tells the kernel the maximum queue size it can cope with, > and if the guest kernel cares about not using insane amounts of > RAM on queues then it should not blindly use the maximum size > but restrict it itself... Yes, this is another way of solving the problem. I think there are three alternatives:
1. Fix the return value of MMIO_QUEUE_NUM_MAX in QEMU (my patch); 2. In guest VM, virtio-mmio shouldn't query MMIO_QUEUE_NUM_MAX. Instead, it should behave similarly to virtio-pci driver which queries VIRTIO_PCI_QUEUE_NUM instead. However this approach requires modification of VIRTIO Specification as VIRTIO_PCI_QUEUE_NUM is WRITE-ONLY in virtio-mmio. 3. Fix virtio_console driver in guest VM. This driver currently takes in info->num and allocates memory based on its value. Apparently we can put a upper-limit on it. Your suggestion could fall in to (2) or (3). Any preference? Thanks, -Wei > > thanks > -- PMM >
