ppoll is not scalable, it has a complexity of O(n). When we have many virtio
queues, the main loop could be slowed down.
epoll, which is O(1), could solve this problem well.
In order to do this, we need to factor out an interface between main loop and
iohandler. What we have now is not good, so patch 1 changed it to GSource,
which is attached to the main context. The posix implementation, which is
identical, attaches each iohandler fd to the iohandler GSrouce. They are
automatically polled by g_poll, or extracted to ppoll.
Patch 2 adds another GSource which attaches a single epoll fd. The epoll fd set
manages all the iohandler fds. Each time the epoll fd is poked by main loop
polling, we call epoll_wait on it to dispatch the fds that are ready to
read/write.
A pitfall is that certain type(s) of fds can't be added to a epoll set. One
such case is the normal file or here document stdin fd. We have to special case
it (where epoll_ctl returns -1 with errno equals to EPERM), and always notify
these fds in the dispatch function.
To show the difference, let's compare the iodepth=1 read test on virtio-blk
by toggling the virtio-serial module (which adds or removes tens of
ioeventfds).
Before:
case rw bs bw iops latency
----------------------------------------------------------------------
vserial=on read 64k 1346 21548 45
vserial=off read 64k 1956 31305 30
After:
case rw bs bw iops latency
----------------------------------------------------------------------
vserial=on read 64k 1727 27647 34
vserial=off read 64k 1741 27868 34
The vserial=on case is better than before because we turned "ppoll with many
fds" into ("ppoll with a few fds" + "epoll with a few fds").
The vserial=off case is worse than before because we turned "ppoll with a few
fds" into ("ppoll with a few fds" + "epoll with a few fds").
To be optimal in both cases, we may dynamically enable/disable epoll depending
on the active ioeventfds. But that's not done yet.
Fam
Fam Zheng (2):
iohandler: Convert I/O handler to GSource
iohandler: Add Linux implementation of iohandler GSource
Makefile.objs | 2 +
include/qemu/iohandler.h | 65 +++++++++++++++
include/qemu/main-loop.h | 2 -
iohandler-linux.c | 213 +++++++++++++++++++++++++++++++++++++++++++++++
iohandler-posix.c | 150 +++++++++++++++++++++++++++++++++
iohandler.c | 90 ++------------------
main-loop.c | 6 +-
7 files changed, 442 insertions(+), 86 deletions(-)
create mode 100644 include/qemu/iohandler.h
create mode 100644 iohandler-linux.c
create mode 100644 iohandler-posix.c
--
1.9.3
