This series of patches adds the multithreading functionality
to OVS dpdk module. Users are able to create multiple pmd
threads and set their cpu affinity via specifying a cpu mask
string similar to the EAL '-c COREMASK' option. Also, the
number of rx queues for each dpdk interface can be configured.
Due to the fact that other ongoing ovs dpdk development can
significantly affect the ovs dpdk performance. Limited tests
were conducted with this series on top of master. Here are
some results.
1. Spirent: 32 bi-directional udp flows (64 byte) between 2 dpdk
ifaces on same cpu socket.
MASTER:
+-------------------+-------------+-------------+
| stats | port1 //6/1 | port2 //6/2 |
+-------------------+-------------+-------------+
| Rx.SigFrameCount | 451719219 | 451716683 |
| Rx.MinFrameLength | 64 | 64 |
| Rx.MaxFrameLength | 198 | 198 |
+-------------------+-------------+-------------+
pktgen: INFO: TX results summary
+--------------------+---------------+---------------+
| stats | port1 | port2 |
+--------------------+---------------+---------------+
| Tx.ParentPort | 917 | 1666 |
| Tx.TotalFrameCount | 1487311447 | 1487318647 |
| Tx.TotalBitCount | 761503460864 | 761507147264 |
| Tx.L1BitCount | 999473292384 | 999478130784 |
| Tx.TxDuration | 100.0 | 100.0 |
| Tx.StreamBlock | 32 udp 64 | 32 udp 64-2 |
| Tx.Load | 100 | 100 |
| Tx.Saturation | 99.9473292384 | 99.9478130784 |
| Tx.DropRate | 69.6286303355 | 69.6286303355 |
+--------------------+---------------+---------------+
3 PMD THREADS ON EACH CPU SOCKET, 6 RX QUEUES PER DPDK IFACE:
+-------------------+-------------+-------------+
| stats | port1 //6/1 | port2 //6/2 |
+-------------------+-------------+-------------+
| Rx.SigFrameCount | 1136484925 | 1140549704 |
| Rx.MinFrameLength | 64 | 64 |
| Rx.MaxFrameLength | 65529 | 198 |
+-------------------+-------------+-------------+
pktgen: INFO: TX results summary
+--------------------+---------------+---------------+
| stats | port1 | port2 |
+--------------------+---------------+---------------+
| Tx.ParentPort | 917 | 1666 |
| Tx.TotalFrameCount | 1488094078 | 1488094078 |
| Tx.TotalBitCount | 761904167936 | 761904167936 |
| Tx.L1BitCount | 999999220416 | 999999220416 |
| Tx.TxDuration | 100.0 | 100.0 |
| Tx.StreamBlock | 32 udp 64 | 32 udp 64-2 |
| Tx.Load | 100 | 100 |
| Tx.Saturation | 99.9999220416 | 99.9999220416 |
| Tx.DropRate | 23.4915768209 | 23.4915768209 |
+--------------------+---------------+---------------+
2. Spirent: 32 bi-directional udp flows (64 byte) between 2 dpdk
ifaces on different cpu socket.
MASTER:
+-------------------+-------------+-------------+
| stats | port1 //6/1 | port2 //6/2 |
+-------------------+-------------+-------------+
| Rx.SigFrameCount | 326922953 | 326923196 |
| Rx.MinFrameLength | 64 | 64 |
| Rx.MaxFrameLength | 16492 | 198 |
+-------------------+-------------+-------------+
pktgen: INFO: TX results summary
+--------------------+---------------+---------------+
| stats | port1 | port2 |
+--------------------+---------------+---------------+
| Tx.ParentPort | 917 | 1666 |
| Tx.TotalFrameCount | 1488094078 | 1488094078 |
| Tx.TotalBitCount | 761904167936 | 761904167936 |
| Tx.L1BitCount | 999999220416 | 999999220416 |
| Tx.TxDuration | 100.0 | 100.0 |
| Tx.StreamBlock | 32 udp 64 | 32 udp 64-2 |
| Tx.Load | 100 | 100 |
| Tx.Saturation | 99.9999220416 | 99.9999220416 |
| Tx.DropRate | 78.0307522667 | 78.0307522667 |
+--------------------+---------------+---------------+
3 PMD THREADS ON EACH CPU SOCKET, 6 RX QUEUES PER DPDK IFACE:
+-------------------+-------------+-------------+
| stats | port1 //6/1 | port2 //6/2 |
+-------------------+-------------+-------------+
| Rx.SigFrameCount | 1487802957 | 1487807151 |
| Rx.MinFrameLength | 12 | 64 |
| Rx.MaxFrameLength | 198 | 198 |
+-------------------+-------------+-------------+
pktgen: INFO: TX results summary
+--------------------+-------------------+-------------------+
| stats | port1 | port2 |
+--------------------+-------------------+-------------------+
| Tx.ParentPort | 917 | 1666 |
| Tx.TotalFrameCount | 1487811649 | 1487807527 |
| Tx.TotalBitCount | 761759564288 | 761757453824 |
| Tx.L1BitCount | 999809428128 | 999806658144 |
| Tx.TxDuration | 100.0 | 100.0 |
| Tx.StreamBlock | 32 udp 64 | 32 udp 64-2 |
| Tx.Load | 100 | 100 |
| Tx.Saturation | 99.9809428128 | 99.9806658144 |
| Tx.DropRate | 0.000304743297568 | 0.000304743297568 |
+--------------------+-------------------+-------------------+
Alex Wang (8):
ovs-numa: Add ovs-numa.{c,h} for extracting and storing cpu socket
and cpu core info.
netdev-dpdk: Add function for getting the socket_id of netdev-dpdk.
netdev-dpdk: Make memory pool name contain the socket id.
netdev-dpdk: Create 'number of dpdk ifaces on same cpu socket' rx
queues and 'number of cpu cores' tx queues for each dpdk interface.
dpif-netdev: Create 'number of dpdk ifaces on cpu socket' pmd
threads for each cpu socket.
netdev-dpdk: Add function for configuring rx queues for all dpdk
interfaces.
ovs-numa: Add support for cpu-mask configuration.
vswitchd: Add option to configure cpu-mask and number of dpdk
rx_queues.
lib/automake.mk | 2 +
lib/dpif-linux.c | 1 +
lib/dpif-netdev.c | 354 +++++++++++++++++++++++++++++++++++++-------
lib/dpif-netdev.h | 3 +-
lib/dpif-provider.h | 7 +
lib/dpif.c | 17 +++
lib/dpif.h | 1 +
lib/netdev-dpdk.c | 150 ++++++++++++++++---
lib/netdev-dpdk.h | 30 ++++
lib/ovs-numa.c | 351 +++++++++++++++++++++++++++++++++++++++++++
lib/ovs-numa.h | 118 +++++++++++++++
ofproto/ofproto-dpif.c | 2 +
ofproto/ofproto-provider.h | 6 +
ofproto/ofproto.c | 16 ++
ofproto/ofproto.h | 2 +
tests/ofproto-macros.at | 1 +
vswitchd/bridge.c | 5 +
vswitchd/vswitch.xml | 29 ++++
18 files changed, 1017 insertions(+), 78 deletions(-)
create mode 100644 lib/ovs-numa.c
create mode 100644 lib/ovs-numa.h
--
1.7.9.5
_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev
