On Thu, Feb 21, 2013 at 08:54:44PM +0800, Liu Ping Fan wrote: > This is a emulation to virtio-blk dataplane, which push the data > handling out of biglock. And it is a try to implement this process > in userspace, while vhost-net in kernel. > > The iperf's result show it improving the perfermance of base line, > but still slower than vhost-net (maybe the rx path need optimized). > > Todo: > implement fine lock for net-subsystem > > Liu Ping Fan (9): > vring: split the modification and publish of used ring > vring: introduce vring_restore() to restore from img > event poll: make epoll work for normal fd > event poll: pass event type to event callback > event poll: enable event poll handle more than one event each time > virtio net: introduce dataplane for virtio net > tap: make tap peer work on dedicated data-plane thread > virtio net: enable dataplane for virtio net > configure: make virtio net dataplane configurable > > configure | 12 ++ > hw/dataplane/Makefile.objs | 4 +- > hw/dataplane/event-poll.c | 69 +++++-- > hw/dataplane/event-poll.h | 14 ++- > hw/dataplane/virtio-blk.c | 8 +- > hw/dataplane/virtio-net.c | 444 > ++++++++++++++++++++++++++++++++++++++++++++ > hw/dataplane/virtio-net.h | 32 ++++ > hw/dataplane/vring.c | 37 ++++ > hw/dataplane/vring.h | 3 + > hw/virtio-net.c | 94 +++++----- > hw/virtio-net.h | 64 +++++++ > hw/virtio-pci.c | 2 +- > include/net/tap.h | 6 + > net/tap.c | 92 +++++++++- > 14 files changed, 806 insertions(+), 75 deletions(-) > create mode 100644 hw/dataplane/virtio-net.c > create mode 100644 hw/dataplane/virtio-net.h
Interesting patch series. I want to share my thoughts on the status of dataplane and what I'm currently working on. There will probably be some overlap that we can coordinate. First, I want to eventually delete hw/dataplane/ :). That is because dataplane duplicates an event loop, thread-friendly guest RAM access, and virtio. QEMU already has all this functionality except it's not thread-friendly. Because hw/dataplane/ will eventually go away we should avoid adding new code where possible. This especially means things like adding live migration support or solving other limitations. The limitations are there to motivate core QEMU refactoring, so that core QEMU code can be used directly instead of reimplementing it. Here's my TODO list: * Thread pool <-> AioContext The thread pool uses an EventNotifier to invoke callbacks. In a multiple event loop world we need to invoke callbacks in the event loop that they are associated with. Worker threads are also started using a BH so that the thread inherits the iothread CPU affinity and other characteristics. I think this can stay. * CoQueue <-> AioContext (for the scheduling bottom half) CoQueue wakes up coroutines from a BH so that the coroutine that asked to wake them up can run to completion or yield first. We need to wake coroutines in the event loop that they belong to. * BlockDriverState <-> AioContext It probably makes sense to bind a BlockDriverState to an AioContext in which its file descriptors and aio activity happens. * bdrv_drain_all()/bdrv_flush_all() safety against concurrent requests The monitor, live migration, and other components use bdrv_drain_all() to complete all requests. It also prevents new requests from starting since the global mutex is held and no device emulation can run. We need to support equivalent semantics in a world where BlockDriverState may be accessed from another thread and have in-flight aio in that event loop. Once these tasks are complete it should be possible to use BlockDriverState in virtio-blk dataplane. This will mean: * Image format support * Block job support (stream, mirror, commit) * I/O throttling support Then the next steps are: * Thread-friendly guest RAM access API * Convert hw/virtio.c (if necessary) to use thread-friendly RAM access * Assigning ioeventfd to a dedicated thread with its own AioContext event loop * Thread-friendly interrupt API This should finally allow hw/dataplane/ to be dropped. At this point we get live migration and hotplug support back too. Then dataplane mode can become the default and the experimental x-data-plane=on|off option can be dropped. Stefan
