2016-08-08 7:18 GMT-07:00 Loftus, Ciara <ciara.lof...@intel.com>: > > > > The patch mostly looks good to me, thanks. > > I'm not 100% sure about the interface. Can we make the flag interface > > specific? > > I'm not 100% sure about making the flag interface specific :) Do you think > there's a use case for both client and server mode ports to be used in > conjunction with each other? >
Well, I don't have any specific use case in mind :-). I just think it's cleaner making it per interface for two reasons: * I'd like to provide the user with the maximum flexibility that the API allows. I don't like adding artificial limitations, especially in user interfaces, since those are supposed to be stable. * The behavior of an interface depends on the status of the switch. It's like having a global variable that influences the behavior of all the functions. > > > If I'm not mistaken we currently limit vhost-sock-dir to be under OVS > > rundir. With client mode this is not necessary anymore. > > Correct I've fixed this in the next version. Thanks. > > > I hope that client will be made the default mode at some point, I think > we > > should keep that in mind when considering the interface. > > I agree. I think we should wait until at least the QEMU v2.7.0 release > though. > > > Since we're planning to break compatibility with the dpdk phy naming > > change, maybe we can break compatibility also with vhost ports and add a > > path option. > > Ok. So something like this? > > ovs-vsctl add-port vhost0 > ovs-vsctl set Interface vhost0 options:vhost-path=/tmp/v0.sock Maybe we can rely on the presence of this attribute to discern between client and server ports (I would call it vhost-server-path). > Maybe something for a separate standalone patch? > If we're going with the per-interface configuration I think this should be done at the same time. Thanks, Daniele > Thanks, > Ciara > > > > > Thoughts? > > Daniele > > > > 2016-08-04 7:09 GMT-07:00 Ciara Loftus <ciara.lof...@intel.com>: > > A new other_config DB option has been added called 'vhost-driver-mode'. > > By default this is set to 'server' which is the mode of operation OVS > > with DPDK has used up until this point - whereby OVS creates and manages > > vHost user sockets. > > > > If set to 'client', OVS will act as the vHost client and connect to > > sockets created and managed by QEMU which acts as the server. This mode > > allows for reconnect capability, which allows vHost ports to resume > > normal connectivity in event of switch reset. > > > > QEMU v2.7.0+ is required when using OVS in client mode and QEMU in > > server mode. > > > > Signed-off-by: Ciara Loftus <ciara.lof...@intel.com> > > --- > > v2 > > - Updated comments in vhost construct & destruct > > - Add check for server-mode before printing error when destruct is called > > on a running VM > > - Fixed coding style/standards issues > > - Use strcmp instead of strncmp when processing 'vhost-driver-mode' > > > > INSTALL.DPDK-ADVANCED.md | 27 +++++++++++++++++++++++++++ > > NEWS | 1 + > > lib/netdev-dpdk.c | 31 +++++++++++++++++++++++-------- > > vswitchd/vswitch.xml | 13 +++++++++++++ > > 4 files changed, 64 insertions(+), 8 deletions(-) > > > > diff --git a/INSTALL.DPDK-ADVANCED.md b/INSTALL.DPDK-ADVANCED.md > > index f9587b5..a773533 100755 > > --- a/INSTALL.DPDK-ADVANCED.md > > +++ b/INSTALL.DPDK-ADVANCED.md > > @@ -483,6 +483,33 @@ For users wanting to do packet forwarding using > > kernel stack below are the steps > > where `-L`: Changes the numbers of channels of the specified > network > > device > > and `combined`: Changes the number of multi-purpose channels. > > > > + 4. Enable OVS vHost client-mode & vHost reconnect (OPTIONAL) > > + > > + By default, OVS DPDK acts as the vHost socket server and QEMU the > > + client. In QEMU v2.7 the option is available for QEMU to act as > the > > + server. In order for this to work, OVS DPDK must be switched to > 'client' > > + mode. This is possible by setting the 'vhost-driver-mode' DB > entry to > > + 'client' like so: > > + > > + ``` > > + ovs-vsctl set Open_vSwitch . other_config:vhost-driver- > mode="client" > > + ``` > > + > > + This must be done before the switch is launched. It cannot > sucessfully > > + be changed after switch has launched. > > + > > + One must also append ',server' to the 'chardev' arguments on the > > QEMU > > + command line, to instruct QEMU to use vHost server mode, like so: > > + > > + ```` > > + -chardev > > socket,id=char0,path=/usr/local/var/run/openvswitch/vhost0,server > > + ```` > > + > > + One benefit of using this mode is the ability for vHost ports to > > + 'reconnect' in event of the switch crashing or being brought > down. Once > > + it is brought back up, the vHost ports will reconnect > automatically and > > + normal service will resume. > > + > > - VM Configuration with libvirt > > > > * change the user/group, access control policty and restart > libvirtd. > > diff --git a/NEWS b/NEWS > > index 9f09e1c..99412ba 100644 > > --- a/NEWS > > +++ b/NEWS > > @@ -70,6 +70,7 @@ Post-v2.5.0 > > fragmentation or NAT support yet) > > * Support for DPDK 16.07 > > * Remove dpdkvhostcuse port type. > > + * OVS client mode for vHost and vHost reconnect (Requires QEMU 2.7) > > - Increase number of registers to 16. > > - ovs-benchmark: This utility has been removed due to lack of use and > > bitrot. > > diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c > > index 7692cc8..39c448b 100644 > > --- a/lib/netdev-dpdk.c > > +++ b/lib/netdev-dpdk.c > > @@ -136,7 +136,8 @@ BUILD_ASSERT_DECL((MAX_NB_MBUF / > > ROUND_DOWN_POW2(MAX_NB_MBUF/MIN_NB_MBUF)) > > #define OVS_VHOST_QUEUE_DISABLED (-2) /* Queue was disabled by > > guest and not > > * yet mapped to another > queue. */ > > > > -static char *vhost_sock_dir = NULL; /* Location of vhost-user sockets > */ > > +static char *vhost_sock_dir = NULL; /* Location of vhost-user > sockets */ > > +static uint64_t vhost_driver_flags = 0; /* Denote whether client/server > > mode */ > > > > #define VHOST_ENQ_RETRY_NUM 8 > > #define IF_NAME_SZ (PATH_MAX > IFNAMSIZ ? PATH_MAX : IFNAMSIZ) > > @@ -833,7 +834,6 @@ netdev_dpdk_vhost_user_construct(struct netdev > > *netdev) > > struct netdev_dpdk *dev = netdev_dpdk_cast(netdev); > > const char *name = netdev->name; > > int err; > > - uint64_t flags = 0; > > > > /* 'name' is appended to 'vhost_sock_dir' and used to create a > socket in > > * the file system. '/' or '\' would traverse directories, so > they're not > > @@ -856,14 +856,17 @@ netdev_dpdk_vhost_user_construct(struct netdev > > *netdev) > > snprintf(dev->vhost_id, sizeof(dev->vhost_id), "%s/%s", > > vhost_sock_dir, name); > > > > - err = rte_vhost_driver_register(dev->vhost_id, flags); > > + err = rte_vhost_driver_register(dev->vhost_id, vhost_driver_flags); > > if (err) { > > VLOG_ERR("vhost-user socket device setup failure for socket > %s\n", > > dev->vhost_id); > > } else { > > - fatal_signal_add_file_to_unlink(dev->vhost_id); > > - VLOG_INFO("Socket %s created for vhost-user port %s\n", > > - dev->vhost_id, name); > > + if (!(vhost_driver_flags & RTE_VHOST_USER_CLIENT)) { > > + /* OVS server mode - add this socket to list for deletion */ > > + fatal_signal_add_file_to_unlink(dev->vhost_id); > > + VLOG_INFO("Socket %s created for vhost-user port %s\n", > > + dev->vhost_id, name); > > + } > > err = netdev_dpdk_init(netdev, -1, DPDK_DEV_VHOST); > > } > > > > @@ -917,7 +920,8 @@ netdev_dpdk_vhost_destruct(struct netdev > > *netdev) > > struct netdev_dpdk *dev = netdev_dpdk_cast(netdev); > > > > /* Guest becomes an orphan if still attached. */ > > - if (netdev_dpdk_get_vid(dev) >= 0) { > > + if (netdev_dpdk_get_vid(dev) >= 0 > > + && !(vhost_driver_flags & RTE_VHOST_USER_CLIENT)) { > > VLOG_ERR("Removing port '%s' while vhost device still > attached.", > > netdev->name); > > VLOG_ERR("To restore connectivity after re-adding of port, VM on > > socket" > > @@ -927,7 +931,8 @@ netdev_dpdk_vhost_destruct(struct netdev > > *netdev) > > > > if (rte_vhost_driver_unregister(dev->vhost_id)) { > > VLOG_ERR("Unable to remove vhost-user socket %s", > dev->vhost_id); > > - } else { > > + } else if (!(vhost_driver_flags & RTE_VHOST_USER_CLIENT)) { > > + /* OVS server mode - remove this socket from list for deletion > */ > > fatal_signal_remove_file_to_unlink(dev->vhost_id); > > } > > > > @@ -3181,6 +3186,7 @@ dpdk_init__(const struct smap *ovs_other_config) > > int err = 0; > > cpu_set_t cpuset; > > char *sock_dir_subcomponent; > > + const char *val; > > > > if (!smap_get_bool(ovs_other_config, "dpdk-init", false)) { > > VLOG_INFO("DPDK Disabled - to change this requires a > restart.\n"); > > @@ -3211,6 +3217,15 @@ dpdk_init__(const struct smap > > *ovs_other_config) > > } else { > > vhost_sock_dir = sock_dir_subcomponent; > > } > > + val = smap_get(ovs_other_config, "vhost-driver-mode"); > > + if (!val || strcmp(val, "client")) { > > + /* default case */ > > + vhost_driver_flags &= ~RTE_VHOST_USER_CLIENT; > > + VLOG_INFO("OVS server mode (QEMU client mode) selected for > > vHost"); > > + } else { > > + vhost_driver_flags |= RTE_VHOST_USER_CLIENT; > > + VLOG_INFO("OVS client mode (QEMU server mode) selected for > > vHost"); > > + } > > > > argv = grow_argv(&argv, 0, 1); > > argc = 1; > > diff --git a/vswitchd/vswitch.xml b/vswitchd/vswitch.xml > > index 63f0d89..41807b3 100644 > > --- a/vswitchd/vswitch.xml > > +++ b/vswitchd/vswitch.xml > > @@ -299,6 +299,19 @@ > > </p> > > </column> > > > > + <column name="other_config" key="vhost-driver-mode" > > + type='{"type": "string", "enum": ["set", ["server", > "client"]]}'> > > + <p> > > + Specifies which mode OVS will use for vHost. In 'server' > mode, OVS > > + creates and destroys the vHost User sockets. In 'client' > mode, OVS > > + attaches to sockets created by QEMU. > > + </p> > > + <p> > > + Defaults to 'server' mode. Changing this value requires > restarting > > + the daemon. > > + </p> > > + </column> > > + > > <column name="other_config" key="n-handler-threads" > > type='{"type": "integer", "minInteger": 1}'> > > <p> > > -- > > 2.4.3 > > > > _______________________________________________ > > dev mailing list > > dev@openvswitch.org > > http://openvswitch.org/mailman/listinfo/dev > > _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev