v3: * Drop Patch 1 to batch guest notify for non-dataplane The Linux AIO completion BH and the virtio-blk batch notify BH changed order in the AioContext->first_bh list as a side-effect of moving the BH from hw/block/dataplane/virtio-blk.c to hw/block/virtio-blk.c. This caused a serious performance regression for both dataplane and non-dataplane.
I've decided not to move the BH in this series and work on a separate solution for making batch notify generic. The remaining patches have been reordered and cleaned up. * See performance data below. v2: * Simplify s->rq live migration [Paolo] * Use more efficient bitmap ops for batch notification [Paolo] * Fix perf regression due to batch notify BH in wrong AioContext [Christian] The virtio_blk guest driver has supported multiple virtqueues since Linux 3.17. This patch series adds multiple virtqueues to QEMU's virtio-blk emulated device. Ming Lei sent patches previously but these were not merged. This series implements virtio-blk multiqueue for QEMU from scratch since the codebase has changed. Live migration support for s->rq was also missing from the previous series and has been added. It's important to note that QEMU's block layer does not support multiqueue yet. Therefore virtio-blk device processes all virtqueues in the same AioContext (IOThread). Further work is necessary to take advantage of multiqueue support in QEMU's block layer once it becomes available. Performance results: Using virtio-blk-pci,num-queues=4 can produce a speed-up but -smp 4 introduces a lot of variance across runs. No pinning was performed. Results show that there is no regression anymore, thanks to dropping the batch notify BH patch. RHEL 7.2 guest on RHEL 7.2 host with 1 vcpu and 1 GB RAM unless otherwise noted. The default configuration of the Linux null_blk driver is used as /dev/vdb. $ cat files/fio.job [global] filename=/dev/vdb ioengine=libaio direct=1 runtime=60 ramp_time=5 gtod_reduce=1 [job1] numjobs=4 iodepth=16 rw=randread bs=4K $ ./analyze.py runs/ Name IOPS Error unpatched-d6550e9ed2 19269820.2 ± 1.36% unpatched-dataplane-d6550e9ed2 22351400.4 ± 1.07% v3-dataplane 22318511.2 ± 0.77% v3-no-dataplane 18936103.8 ± 1.12% v3-queues-4-no-dataplane 19177021.8 ± 1.45% v3-smp-4-no-dataplane 25509585.2 ± 29.50% v3-smp-4-no-dataplane-no-mq 12466177.2 ± 7.88% Configuration: Name Patched? Dataplane? SMP? MQ? unpatched-d6550e9ed2 N N N N unpatched-dataplane-d6550e9ed2 N Y N N v3-dataplane Y Y N N v3-no-dataplane Y N N N v3-queues-4-no-dataplane Y N N Y v3-smp-4-no-dataplane Y N Y Y v3-smp-4-no-dataplane-no-mq Y N Y N SMP means -smp 4. MQ means virtio-blk-pci,num-queues=4. Stefan Hajnoczi (7): virtio-blk: add VirtIOBlockConf->num_queues virtio-blk: multiqueue batch notify virtio-blk: tell dataplane which vq to notify virtio-blk: associate request with a virtqueue virtio-blk: live migrate s->rq with multiqueue virtio-blk: dataplane multiqueue support virtio-blk: add num-queues device property hw/block/dataplane/virtio-blk.c | 81 +++++++++++++++++++++++++++++------------ hw/block/dataplane/virtio-blk.h | 2 +- hw/block/virtio-blk.c | 52 +++++++++++++++++++++----- include/hw/virtio/virtio-blk.h | 6 ++- 4 files changed, 105 insertions(+), 36 deletions(-) -- 2.5.5
