On 12/01/2016 08:26 PM, Stefan Hajnoczi wrote: > v4: > * Added poll time self-tuning algorithm [Christian and Paolo] > * Try a single iteration of polling to avoid non-blocking > ppoll(2)/epoll_wait(2) [Paolo] > * Reordered patches to make performance analysis easier - see below
This looks quite promising. I will give this to our performance folks to get some more controlled measurements. > > v3: > * Avoid ppoll(2)/epoll_wait(2) if polling succeeded [Paolo] > * Disable guest->host virtqueue notification during polling [Christian] > * Rebased on top of my virtio-blk/scsi virtqueue notification disable patches > > v2: > * Uninitialized node->deleted gone [Fam] > * Removed 1024 polling loop iteration qemu_clock_get_ns() optimization which > created a weird step pattern [Fam] > * Unified with AioHandler, dropped AioPollHandler struct [Paolo] > (actually I think Paolo had more in mind but this is the first step) > * Only poll when all event loop resources support it [Paolo] > * Added run_poll_handlers_begin/end trace events for perf analysis > * Sorry, Christian, no virtqueue kick suppression yet > > Recent performance investigation work done by Karl Rister shows that the > guest->host notification takes around 20 us. This is more than the "overhead" > of QEMU itself (e.g. block layer). > > One way to avoid the costly exit is to use polling instead of notification. > The main drawback of polling is that it consumes CPU resources. In order to > benefit performance the host must have extra CPU cycles available on physical > CPUs that aren't used by the guest. > > This is an experimental AioContext polling implementation. It adds a polling > callback into the event loop. Polling functions are implemented for > virtio-blk > virtqueue guest->host kick and Linux AIO completion. > > The -object iothread,poll-max-ns=NUM parameter sets the number of nanoseconds > to poll before entering the usual blocking poll(2) syscall. Try setting this > parameter to the time from old request completion to new virtqueue kick. By > default no polling is done so you must set this parameter to get busy polling. > > Patch series overview: > > Here are convenient points in the patch series at which to do performance > benchmarking to see how various pieces interact. > > 1. Basic -iothread poll-max-ns=NUM parameter without self-tuning: > > aio: add flag to skip fds to aio_dispatch() > aio: add AioPollFn and io_poll() interface > aio: add polling mode to AioContext > virtio: poll virtqueues for new buffers > linux-aio: poll ring for completions > iothread: add polling parameters > > 2. My "[PATCH 0/3] virtio: disable notifications in blk and scsi" series is > needed as a base. This is unrelated to polling but we'll also disable > notifications during polling later: > > virtio-blk: suppress virtqueue kick during processing > virtio-scsi: suppress virtqueue kick during processing > > 3. Disable virtqueue notifications (reduce vmexits) during polling: > > virtio: turn vq->notification into a nested counter > aio: add .io_poll_begin/end() callbacks > virtio: disable virtqueue notifications during polling > > 4. Self-tuning, adjusts polling time depending on wait times: > > aio: self-tune polling time > iothread: add poll-grow and poll-shrink parameters > > Stefan Hajnoczi (13): > aio: add flag to skip fds to aio_dispatch() > aio: add AioPollFn and io_poll() interface > aio: add polling mode to AioContext > virtio: poll virtqueues for new buffers > linux-aio: poll ring for completions > iothread: add polling parameters > virtio-blk: suppress virtqueue kick during processing > virtio-scsi: suppress virtqueue kick during processing > virtio: turn vq->notification into a nested counter > aio: add .io_poll_begin/end() callbacks > virtio: disable virtqueue notifications during polling > aio: self-tune polling time > iothread: add poll-grow and poll-shrink parameters > > include/block/aio.h | 53 +++++++- > include/sysemu/iothread.h | 5 + > aio-posix.c | 308 > +++++++++++++++++++++++++++++++++++++++----- > aio-win32.c | 32 ++++- > async.c | 21 ++- > block/curl.c | 8 +- > block/iscsi.c | 3 +- > block/linux-aio.c | 19 ++- > block/nbd-client.c | 8 +- > block/nfs.c | 7 +- > block/sheepdog.c | 26 ++-- > block/ssh.c | 4 +- > block/win32-aio.c | 4 +- > hw/block/virtio-blk.c | 18 ++- > hw/scsi/virtio-scsi.c | 36 +++--- > hw/virtio/virtio.c | 54 ++++++-- > iohandler.c | 2 +- > iothread.c | 84 ++++++++++++ > nbd/server.c | 9 +- > stubs/set-fd-handler.c | 1 + > tests/test-aio.c | 4 +- > util/event_notifier-posix.c | 2 +- > trace-events | 6 + > 23 files changed, 604 insertions(+), 110 deletions(-) >
