Adding a self-guide for configuring native userspace tunneling in OVS with/without DPDK ports. This document also provides necessary debugging commands to identify and resolve the userspace tunneling issues in OVS.
Signed-off-by: Sugesh Chandran <sugesh.chand...@intel.com> --- README-native-tunneling-DPDK.md | 206 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 206 insertions(+) create mode 100644 README-native-tunneling-DPDK.md diff --git a/README-native-tunneling-DPDK.md b/README-native-tunneling-DPDK.md new file mode 100644 index 0000000..47609b5 --- /dev/null +++ b/README-native-tunneling-DPDK.md @@ -0,0 +1,206 @@ +Openvswitch Native tunneling configuration guide +======================================================= + +This guide is for configuring userspace tunneling in Open-vSwitch. The +traditional OVS with kernel datapath uses kernel module to perform the tunneling +however this setup performs all the tunneling operations purely in the +userspace. This way userspace-tunnelling is platform independent. + +This setup needs an additional bridge called “br-phy1” when compared to the +kernel based OVS. The purpose of this bridge is to make available the kernel +network stack for routing and arp resolution. The data path needs to look-up +the routing table and arp table to prepare the tunnel header and xmit data to +the output port. + +The tunneling setup is found as below: + + +--------------+ + | VM-0 | 192.168.1.1/24 + +--------------+ + (vm_port0) + | + | + | + | + | + +--------------+ + | br-int | 192.168.1.2/24 + +--------------+ +--------------+ + | vxlan0 | | vxlan0 | + +--------------+ +--------------+ + | | + | | + | | + | | + | | + | | + 172.168.1.1/24 | + +--------------+ | + | br-phy1 | 172.168.1.2/24 + +--------------+ +---------------+ + | dpdk0/eth1 |==========================| eth1 | + +--------------+ +---------------+ + Host A with OVS. Remote host. + +#### Prerequisites +The host must be pre-installed with OVS-DPDK, QEMU and virtual machine images. +Please refer the installation guide of relevant modules for these instructions. +ovs-vswitchd and OVSDB server must be up and running before proceeding to any +of the configuration steps. + +This setup guide covers only the required steps for setting up VxLAN userspace +tunneling. The same approach can be used for any other tunneling protocols, by +specifying the appropriate tunneling protocol type. + +Note:- This configuration guide is for setting up the VxLAN tunneling on one +host(local host). The same steps have to perform on the remote host as well for +setting up a VM<->VM VxLAN tunnel setup. The only difference for setting up the +userspace-tunneling in remote node is , VM and VxLAN tunnel ip addresses are +different. + +#### Configuration steps +1. Create the “br-int” bridge as below, + + ***HOST-A$ sudo ovs-vsctl --may-exist add-br br-int -- set Bridge br-int datapath_type=netdev -- br-set-external-id br-int bridge-id br-int -- set bridge br-int fail-mode=standalone*** + +2. Add the VM interface to the “br-int”. + + ***HOST-A$ sudo ovs-vsctl add-port br-int vm_port0 -- set Interface vm_port0 type=dpdkvhostuser*** + + `[“vm_port0” is the vhost-user interface name for the VM0. This interface name +must be used while qemu starting the virtual machine.]` + +3. Start the VM(VM-0) using vhost-user network backend. The vhost-user +interface name is "vm_port0". + +4. Set the Ip address 192.168.1.1 to the VM interface. Run the following +command inside the VM to set the Ip address. + + ***VM-0$ sudo ip addr add 192.168.1.1/24 dev eth0*** + + `[“eth0” is the interface inside VM. its possible to set the ip address using +"ifconfig" command as "sudo ifconfig eth0 192.168.1.1/24". Please use either +one command to set the ip address.]` + +5. Attach the VxLAN tunnel interface to the “br-int“ bridge. + + ***HOST-A$ sudo ovs-vsctl add-port br-int vxlan0 -- set interface vxlan0 type=vxlan options:remote_ip=172.168.1.2*** + + `[“172.168.1.2” is the remote tunnel end point address, On remote host this +will be 172.168.1.1.]` + +6. Create the “br-phy1” bridge for attaching the physical interface. + + ***HOST-A$ sudo ovs-vsctl --may-exist add-br br-phy1 -- set Bridge br-phy1 datapath_type=netdev -- br-set-external-id br-phy1 bridge-id br-phy1 -- set bridge br-phy1 fail-mode=standalone other_config:hwaddr="<mac address of eth1 interface>"*** + + `[“eth1” is the physical interface on the host, Assign mac address of “eth1” +to the “br-phy1”.]` + +7. The physical port "eth1" can operate either as a KNI(kernel network +interface) or a DPDK interface. Depending on the operating mode, attach "eth1" +to the "br-phy1" bridge as follows. + + Use step-7 if "eth1" is KNI. Please follow Step :8 rather than this step in +case "eth1" is a DPDK Interface.This step will cause to loose the connectivity +through “eth1” (refer configuration problems in https://github.com/openvswitch/ovs/blob/master/FAQ.md for more details) for a +while. The connectivity can be restored by moving the IP address to the +“br-phy1” internal interface. The following command-set will do that, + + ***HOST-A$ sudo ovs-vsctl --timeout 10 add-port br-phy1 eth1*** + + ***HOST-A$ sudo ip addr add 172.168.1.1/24 dev br-phy1*** + + ***HOST-A$ sudo ip link set br-phy1 up*** + + ***HOST-A$ sudo ip addr flush dev eth1 2>/dev/null*** + + ***HOST-A$ sudo ip link set eth1 up*** + + ***HOST-A$ sudo iptables –F*** + +8. Steps for attaching “eth1” to “br-phy1” is slightly different in case +“eth1” is a DPDK interface. DPDK interfaces are not managed by the kernel, so +the port details are not visible on any “ip” commands. + + ***HOST-A$ sudo ./tools/dpdk_nic_bind.py --bind=igb_uio eth1*** + + `["dpdk_nic_bind.py" is a utility script to bind/unbind interfaces to DPDK/Linux +kernel.More details on bind/unbind can be found at http://dpdk.org/doc/guides/linux_gsg/build_dpdk.html#binding-and-unbinding-network-ports-to-from-the-kernel-modules]` + + ***HOST-A$ sudo ovs-vsctl --timeout 10 add-port br-phy1 dpdk0 -- set Interface dpdk0 type=dpdk*** + + ***HOST-A$ sudo ip addr add 172.168.1.1/24 dev br-phy1*** + + ***HOST-A$ sudo ip link set br-phy1 up*** + + ***HOST-A$ sudo iptables –F*** + + `[“eth1” is mapped to DPDK port “dpdk0”. DPDK driver assign port names that +starts with “dpdk”]` + +Now the traffic from “VM0” will be VxLAN encapsulated and send out over eth1/dpdk0 +interface. + +TCPDUMP doesnt work on DPDK interfaces(eth0) as its no longer managed by the kernel. + +#### Debugging + +* Verify the created tunnel port details. + + ***HOST-A$ sudo ovs-appctl tnl/ports/show*** + +* As mentioned before, its necessary that the vswitch should have the arp-table +entries to do tunnelling at userspace. The learned arp entries in the vswitch +can be verified by, + + ***HOST-A$ sudo ovs-appctl tnl/arp/show*** + + the arp entries can be flushed using, + + ***HOST-A$ sudo ovs-appctl tnl/arp/flush*** + + To set a specific arp entry, + + ***HOST-A$ sudo ovs-appctl tnl/arp/set <bridge> <ip addr> <mac addr>*** + + In the above test setup, the following arp entries has to set in case they are + not present. + + ***HOST-A$ sudo ovs-appctl tnl/arp/set br-int 172.168.1.1 <mac addr of br-phy1>*** + + ***HOST-A$ sudo ovs-appctl tnl/arp/set br-phy1 172.168.1.2 <mac addr of remote TEP>*** + +* Similarly OVS uses routing table entries to xmit the tunnel packets. The +vswitch routing entries can be verified by, + + ***HOST-A$ sudo ovs-appctl ovs/route/show*** + + To delete the route entries. + + ***HOST-A$ sudo ovs-appctl ovs/route/del*** + +* To lookup a route entry for a destination please use, + + ***HOST-A$ ovs-appctl ovs/route/lookup*** + +* In case the route entries are missing for tunnel packet forwarding, it can be +added using the following command, + + ***HOST-A$ sudo ovs-appctl ovs/route/add 172.168.1.1/24 br-phy1*** + + This command instructs OVS to route all traffic destined for “172.168.1.2” to + bridge “br-phy1” + +* To verify the range of tunneling source ports, + + ***HOST-A$ sudo ovs-appctl tnl/egress_port_range*** + +* To see the configured datapath ports, + + ***HOST-A$ sudo ovs-appctl dpif/show*** + +* To verify the flows programmed on the OVS datapath, + + ***HOST-A$ sudo ovs-appctl dpctl/dump-flows*** + + [This command shows how OVS forwards the packets in datapath.] -- 2.1.4 _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
