On Tue, May 17, 2016 at 12:44 AM, Flavio Leitner <f...@sysclose.org> wrote:
> On Fri, May 13, 2016 at 10:25:41PM +0200, Mauricio Vasquez B wrote: > > In order to use dpdk ports in ovs they have to be bound to a DPDK > > compatible driver before ovs is started. > > > > This patch adds the possibility to hotplug (or hot-unplug) a device > > after ovs has been started. The implementation adds two appctl commands: > > netdev-dpdk/port-attach and netdev-dpdk/port-detach > > > > After the user attaches a new device, it has to be added to a bridge > > using the add-port command, similarly, before detaching a device, > > it has to be removed using the del-port command. > > > > Signed-off-by: Mauricio Vasquez B < > mauricio.vasquezber...@studenti.polito.it> > > The patch looks good, but it didn't work for me with VFIO (it's > not supported by the DPDK hotplug framework). Since the doc > includes instructions to use VFIO, should we add a note for the > lack of hotplug support? > > Yes, we should. I'll wait few days to see if there is more feedback and I'll send a new version including that note. > The attach/detach operations seem to work with uio_pci_generic, > but it doesn't move packets. The interface statistics shows > only 'errs' increasing a lot and few 'drops'. > > Kernel: 4.5.0+ > OVS: master (b396293a) + patch > DPDK: 16.04 > Drivers: vfio-pci or uio_pci_generic > Hardware: 82599ES > > I tried uio_pci_generic in the following conditions without any problem. Kernel: 3.19.0-32-generic OVS: master(d640a7e1) + patch DPDK: 16.04 Hardware: X540-AT2 > Do you know if there are patches/plans to support that with VFIO or > uio_pci_generic? > I am not aware of any plans for it. Thanks for reviewing Flavio! > fbl > > > > --- > > v5: > > - use two appctl commands instead of a single one > > - rebase to master > > v4: > > - fix typo in commit message > > - remove unnecessary whitespace change in INSTALL.DPDK.md > > v3: > > - create dpdk_port_attach and dpdk_port_detach functions > > - modify mutex locking order > > v2: > > - use rte_eth_dev_is_valid_port() to check if a port is valid > > INSTALL.DPDK.md | 24 +++++++++++++ > > NEWS | 1 + > > lib/netdev-dpdk.c | 101 > ++++++++++++++++++++++++++++++++++++++++++++++++++---- > > 3 files changed, 119 insertions(+), 7 deletions(-) > > > > diff --git a/INSTALL.DPDK.md b/INSTALL.DPDK.md > > index 93f92e4..8d5a0e0 100644 > > --- a/INSTALL.DPDK.md > > +++ b/INSTALL.DPDK.md > > @@ -267,6 +267,29 @@ Using the DPDK with ovs-vswitchd: > > For more details regarding egress-policer parameters please refer to > the > > vswitch.xml. > > > > +9. Port Hotplug > > + > > + Ovs supports port hotplugging, it allows to use ports that were not > bound > > + to DPDK when vswitchd was started. > > + In order to attach a port, it has to be bound to DPDK using the > > + dpdk_nic_bind.py script: > > + > > + `$DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio 0000:01:00.0` > > + > > + Then it can be attached to OVS: > > + > > + `ovs-appctl netdev-dpdk/port-attach 0000:01:00.0` > > + > > + At this point, the user can create a ovs port using the add-port > command. > > + > > + It is also possible to detach a port from ovs, the user has to > remove the > > + port using the del-port command, then it can be detached using: > > + > > + `ovs-appctl netdev-dpdk/port-detach dpdk0` > > + > > + This feature is not supported by all the NICs, please refer to the > > + [DPDK Port Hotplug Framework] in order to get more information. > > + > > Performance Tuning: > > ------------------- > > > > @@ -999,3 +1022,4 @@ Please report problems to b...@openvswitch.org. > > [INSTALL.md]:INSTALL.md > > [DPDK Linux GSG]: > http://www.dpdk.org/doc/guides/linux_gsg/build_dpdk.html#binding-and-unbinding-network-ports-to-from-the-igb-uioor-vfio-modules > > [DPDK Docs]: http://dpdk.org/doc > > +[DPDK Port Hotplug Framework]: > http://dpdk.org/doc/guides/prog_guide/port_hotplug_framework.html > > diff --git a/NEWS b/NEWS > > index 4e81cad..d89d9a7 100644 > > --- a/NEWS > > +++ b/NEWS > > @@ -32,6 +32,7 @@ Post-v2.5.0 > > * DB entries have been added for many of the DPDK EAL command line > > arguments. Additional arguments can be passed via the dpdk-extra > > entry. > > + * Port Hotplug is now supported. > > - ovs-benchmark: This utility has been removed due to lack of use and > > bitrot. > > - ovs-appctl: > > diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c > > index af86d19..6159a60 100644 > > --- a/lib/netdev-dpdk.c > > +++ b/lib/netdev-dpdk.c > > @@ -630,7 +630,7 @@ dpdk_eth_dev_init(struct netdev_dpdk *dev) > OVS_REQUIRES(dpdk_mutex) > > int diag; > > int n_rxq, n_txq; > > > > - if (dev->port_id < 0 || dev->port_id >= rte_eth_dev_count()) { > > + if (!rte_eth_dev_is_valid_port(dev->port_id)) { > > return ENODEV; > > } > > > > @@ -2118,6 +2118,83 @@ netdev_dpdk_set_admin_state(struct unixctl_conn > *conn, int argc, > > unixctl_command_reply(conn, "OK"); > > } > > > > +static void > > +netdev_dpdk_port_attach(struct unixctl_conn *conn, int argc OVS_UNUSED, > > + const char *argv[], void *aux OVS_UNUSED) > > +{ > > + int ret; > > + char response[128]; > > + uint8_t port_id; > > + > > + ovs_mutex_lock(&dpdk_mutex); > > + > > + ret = rte_eth_dev_attach(argv[1], &port_id); > > + if (ret < 0) { > > + snprintf(response, sizeof(response), > > + "Error attaching device '%s'", argv[1]); > > + ovs_mutex_unlock(&dpdk_mutex); > > + unixctl_command_reply_error(conn, response); > > + return; > > + } > > + > > + snprintf(response, sizeof(response), > > + "Device '%s' has been attached as 'dpdk%d'", argv[1], > port_id); > > + > > + ovs_mutex_unlock(&dpdk_mutex); > > + unixctl_command_reply(conn, response); > > +} > > + > > +static void > > +netdev_dpdk_port_detach(struct unixctl_conn *conn, int argc OVS_UNUSED, > > + const char *argv[], void *aux OVS_UNUSED) > > +{ > > + int ret; > > + char response[128]; > > + unsigned int parsed_port; > > + uint8_t port_id; > > + char devname[RTE_ETH_NAME_MAX_LEN]; > > + > > + ovs_mutex_lock(&dpdk_mutex); > > + > > + ret = dpdk_dev_parse_name(argv[1], "dpdk", &parsed_port); > > + if (ret) { > > + snprintf(response, sizeof(response), > > + "'%s' is not a valid port", argv[1]); > > + goto error; > > + } > > + > > + port_id = parsed_port; > > + > > + struct netdev *netdev = netdev_from_name(argv[1]); > > + if (netdev) { > > + netdev_close(netdev); > > + snprintf(response, sizeof(response), > > + "Port '%s' is being used. Remove it before detaching", > > + argv[1]); > > + goto error; > > + } > > + > > + rte_eth_dev_close(port_id); > > + > > + ret = rte_eth_dev_detach(port_id, devname); > > + if (ret < 0) { > > + snprintf(response, sizeof(response), > > + "Port '%s' can not be detached", argv[1]); > > + goto error; > > + } > > + > > + snprintf(response, sizeof(response), > > + "Port '%s' has been detached", argv[1]); > > + > > + ovs_mutex_unlock(&dpdk_mutex); > > + unixctl_command_reply(conn, response); > > + return; > > + > > +error: > > + ovs_mutex_unlock(&dpdk_mutex); > > + unixctl_command_reply_error(conn, response); > > +} > > + > > /* > > * Set virtqueue flags so that we do not receive interrupts. > > */ > > @@ -2404,6 +2481,15 @@ dpdk_common_init(void) > > "[netdev] up|down", 1, 2, > > netdev_dpdk_set_admin_state, NULL); > > > > + unixctl_command_register("netdev-dpdk/port-attach", > > + "pci address of device", 1, 1, > > + netdev_dpdk_port_attach, NULL); > > + > > + unixctl_command_register("netdev-dpdk/port-detach", > > + "port", 1, 1, > > + netdev_dpdk_port_detach, NULL); > > + > > + ovs_thread_create("dpdk_watchdog", dpdk_watchdog, NULL); > > } > > > > /* Client Rings */ > > @@ -2414,7 +2500,7 @@ dpdk_ring_create(const char dev_name[], unsigned > int port_no, > > { > > struct dpdk_ring *ivshmem; > > char ring_name[RTE_RING_NAMESIZE]; > > - int err; > > + int err, port_id; > > > > ivshmem = dpdk_rte_mzalloc(sizeof *ivshmem); > > if (ivshmem == NULL) { > > @@ -2448,19 +2534,20 @@ dpdk_ring_create(const char dev_name[], unsigned > int port_no, > > return ENOMEM; > > } > > > > - err = rte_eth_from_rings(dev_name, &ivshmem->cring_rx, 1, > > - &ivshmem->cring_tx, 1, SOCKET0); > > + port_id = rte_eth_from_rings(dev_name, &ivshmem->cring_rx, 1, > > + &ivshmem->cring_tx, 1, SOCKET0); > > > > - if (err < 0) { > > + if (port_id < 0) { > > rte_free(ivshmem); > > return ENODEV; > > } > > > > ivshmem->user_port_id = port_no; > > - ivshmem->eth_port_id = rte_eth_dev_count() - 1; > > + ivshmem->eth_port_id = port_id; > > + *eth_port_id = port_id; > > + > > ovs_list_push_back(&dpdk_ring_list, &ivshmem->list_node); > > > > - *eth_port_id = ivshmem->eth_port_id; > > return 0; > > } > > > > -- > > 1.9.1 > > > > _______________________________________________ > > dev mailing list > > dev@openvswitch.org > > http://openvswitch.org/mailman/listinfo/dev > > -- > fbl > > _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
