Currently we treat set_multiq() calls specially in netdev and dpif-netdev: every pmd thread must be stopped and set_multiq() is allowed to destroy and recreate the device.
I think we can improve this by: * Generalizing the mechanism to allow changing other parameters at runtime (such as MTU). * Involving less the above layer (dpif-netdev). The request for changes often comes from below (netdev_dpdk_set_config(), or the vhost new_device() callback). There's no need for dpif-netdev to remember the requested value, all that it needs to know is that a configuration change is requested. This series implements exactly this: a mechanism to allow a netdev provider to request configuration changes, to which dpif-netdev will respond by stopping rx/tx and calling a netdev function to appy the new configuration. The new mechanism is used in this series to replace the set_multiq() call, but the idea is to use it also at least for: * Changing the MTU at runtime * Automatically detecting the number of rx queues for a vhost-user device * Move a DPDK vhost device to the proper NUMA socket The first commits refactor some code in dpif-netdev and, most importantly avoid using RCU for ports. Each thread will have its local copy of all the ports in the datapath. The series is also available here: https://github.com/ddiproietto/ovs/tree/configchangesv8 v8: * Update comment in rcu.h: ovs_mutex_cond_wait doesn't quiesce. * Change 'set_multiq' to 'set_tx_multiq'. * Added documentation in comments and commit messages explaining thread local port cache. * Fixed style issues reported by checkpatch.py. * Fixed race condition when deleting pmd thread. v7: * Dropped already applied patches. * Stop using RCU for ports. * Rebased against master. v6: * Rebased against master. * Check return value of netdev_rxq_open(). * Fix comment. v5: * Style fixes. * Fixed a bug in dp_netdev_free() in patch 6. v4: * Added another patch to uniform names of variables in netdev-dpdk (no functional change) * Update some netdev comments to document the relation between netdev_set_multiq() and netdev_reconfigure() * Clarify that when netdev_reconfigure() is called no call to netdev_send() or netdev_rxq_recv() must be issued. * Move check to skip reconfiguration in netdev_dpdk_reconfigure() before rte_eth_dev_stop(). v3: * Fixed another outdated comment about rx queue configuration, as pointed out by Mark * Removed unnecessary and buggy initialization of requested_n_rxq in reconfigure_pmd_threads(). * Removed unused 'err' variable in netdev_dpdk_set_multiq(). * Changed comparison in netdev_set_multiq() to use previous 'netdev->requested_n_txq' instead of 'netdev->up.n_txq' * Return immediately in netdev_dpdk_reconfigure() if configuration didn't change anything. v2: * Fixed do_add_port(): we have to call netdev_reconfigure() before opening the rxqs. This prevents memory leaks, and makes sure that the datapath polls the appropriate number of queues * Fixed netdev_dpdk_vhost_set_multiq(): it must call netdev_request_reconfigure(). Since it is now equal to netdev_dpdk_set_multiq(), the two function have been merged. * Fixed netdev_dpdk_set_config(): dev->requested_n_rxq is now accessed while holding the appropriate mutex. * Fixed some outdated comments about rx queue configuration. Daniele Di Proietto (16): dpif-netdev: Destroy 'port_mutex' in dp_netdev_free(). dpif-netdev: Remove unused 'index' in dp_netdev_pmd_thread. dpif-netdev: Factor out port_create() from do_add_port(). dpif-netdev: Add functions to modify rxq without reloading pmd threads. dpif-netdev: Fix race condition in pmd thread initialization. dpif-netdev: Remove duplicate code in dp_netdev_set_pmds_on_numa(). hmap: Add HMAP_FOR_EACH_POP. dpif-netdev: Add pmd thread local port cache for transmission. hmap: Use struct for hmap_at_position(). dpif-netdev: Use hmap for ports. ovs-thread: Do not quiesce in ovs_mutex_cond_wait(). ofproto-dpif: Call dpif_poll_threads_set() before dpif_run(). dpif-netdev: Change pmd thread configuration in dpif_netdev_run(). dpif-netdev: Handle errors in reconfigure_pmd_threads(). netdev: Add reconfigure request mechanism. netdev-dpdk: Use ->reconfigure() call to change rx/tx queues. lib/cfm.c | 5 +- lib/dpif-netdev.c | 702 +++++++++++++++++++++++++++---------------- lib/dpif-provider.h | 3 +- lib/hmap.c | 26 +- lib/hmap.h | 11 +- lib/id-pool.c | 5 +- lib/learning-switch.c | 5 +- lib/netdev-bsd.c | 3 +- lib/netdev-dpdk.c | 194 ++++++------ lib/netdev-dummy.c | 3 +- lib/netdev-linux.c | 8 +- lib/netdev-provider.h | 50 ++- lib/netdev-vport.c | 3 +- lib/netdev.c | 75 +++-- lib/netdev.h | 7 +- lib/odp-util.c | 7 +- lib/ovs-rcu.h | 3 +- lib/ovs-thread.c | 2 - lib/sset.c | 12 +- lib/sset.h | 7 +- ofproto/bond.c | 10 +- ofproto/in-band.c | 5 +- ofproto/ofproto-dpif-ipfix.c | 5 +- ofproto/ofproto-dpif-xlate.c | 5 +- ofproto/ofproto-dpif.c | 12 +- ofproto/ofproto.c | 5 +- ofproto/pinsched.c | 5 +- ovn/controller-vtep/vtep.c | 5 +- ovn/controller/encaps.c | 5 +- ovn/controller/lport.c | 5 +- ovn/controller/ofctrl.c | 5 +- ovn/controller/physical.c | 4 +- ovn/controller/pinctrl.c | 5 +- ovn/lib/expr.c | 5 +- ovn/northd/ovn-northd.c | 10 +- ovsdb/monitor.c | 5 +- ovsdb/row.c | 5 +- tests/library.at | 2 +- tests/test-hmap.c | 42 +++ vswitchd/system-stats.c | 6 + 40 files changed, 785 insertions(+), 497 deletions(-) -- 2.1.4 _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
