A quick status update and request for comments on the future
direction of this work.
Changes since last post (tl;dr)
* Reduced ring size from 4M to 1M
* Added proper handling for socket errors (EPOLLERR)
* More benchmarking
Numbers, first:
Test: * Avg pps as reported by pktgen for a single pktgen thread
* qemu, vcpu=2, mem=2GB, megaflows disabled
Orig [0] ZC [1] mmap [2] shared-ofpbuf [3]
64B (linear) 624K 564K 962K 900K
1.5K (linear) 612K 629K 800K 891K
9K (linear) 476K 474K 508K 513K
9K (4 frags) 365K 411K 503K 631K
[0] git master branch modified to disable zerocopy
[1] git master branch (18a6a6cd9)
A single copy is avoided when constructing message in the
kernel
[2] Patch 1-3 applied
Separate shared memory based rx/tx ring buffer is used to
carry upcall messages. ovs-vswitchd copies data off ring.
[3] Patch 4 applied
ovs-vswitchd reuses the shared memory and avoid another
copy
Numbers on ring buffer size:
ring-size: 512K 1M 2M 4M
pps: 596K 914K 926K 900K
Q1: The gains are significant and can be optimized further. But even
with the ring size reduced from 8Mb to 2MB per socket (rx+tx),
the resource consumption will not allow for a separate ring for
every port * handler.
Options:
1. Reuse same ring for multiple vport
2. Require manual enablement per port
Q2: How have you been testing fairness so far?
Thoughts? Comments?
Thomas Graf (4):
netlink: Support for memory mapped Netlink sockets
nlmon: Enable memory mapped Netlink socket
upcall: Enable memory mapped Netlink sockets
netlink: COW mmap Netlink messages to avoid memcpy()
lib/dpif-linux.c | 7 +-
lib/netlink-protocol.h | 39 ++++++
lib/netlink-socket.c | 369 ++++++++++++++++++++++++++++++++++++++++++++-----
lib/netlink-socket.h | 10 +-
lib/ofpbuf.c | 6 +
lib/ofpbuf.h | 2 +
utilities/nlmon.c | 2 +-
7 files changed, 396 insertions(+), 39 deletions(-)
--
1.8.3.1
_______________________________________________
dev mailing list
[email protected]
http://openvswitch.org/mailman/listinfo/dev
