Thanks for writing this up, it will definitely help many users. I'm not sure the appropriate place for this is the OVS source tree (given that's similar to README-native-tunneling.md), but this seems a perfect candidate for a configuration cookbook (http://openvswitch.org/support/config-cookbooks/).
We keep the openvswitch.org website repository here (https://github.com/openvswitch/openvswitch.github.io), will you consider creating a pull request to add this guide to the configuration cookbooks? I believe that there's no need to rewrite this in HTML, Markdown should be supported. Thanks, Daniele On 16/10/2015 02:32, "Sugesh Chandran" <[email protected]> wrote: >Adding a self-guide for configuring native userspace tunneling in OVS >with/without DPDK ports. This document also provides necessary debugging >commands to identify and resolve the userspace tunneling issues in OVS. > >Signed-off-by: Sugesh Chandran <[email protected]> >--- > README-native-tunneling-DPDK.md | 206 >++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 206 insertions(+) > create mode 100644 README-native-tunneling-DPDK.md > >diff --git a/README-native-tunneling-DPDK.md >b/README-native-tunneling-DPDK.md >new file mode 100644 >index 0000000..47609b5 >--- /dev/null >+++ b/README-native-tunneling-DPDK.md >@@ -0,0 +1,206 @@ >+Openvswitch Native tunneling configuration guide >+======================================================= >+ >+This guide is for configuring userspace tunneling in Open-vSwitch. The >+traditional OVS with kernel datapath uses kernel module to perform the >tunneling >+however this setup performs all the tunneling operations purely in the >+userspace. This way userspace-tunnelling is platform independent. >+ >+This setup needs an additional bridge called ³br-phy1² when compared to >the >+kernel based OVS. The purpose of this bridge is to make available the >kernel >+network stack for routing and arp resolution. The data path needs to >look-up >+the routing table and arp table to prepare the tunnel header and xmit >data to >+the output port. >+ >+The tunneling setup is found as below: >+ >+ +--------------+ >+ | VM-0 | 192.168.1.1/24 >+ +--------------+ >+ (vm_port0) >+ | >+ | >+ | >+ | >+ | >+ +--------------+ >+ | br-int | 192.168.1.2/24 >+ +--------------+ +--------------+ >+ | vxlan0 | | vxlan0 | >+ +--------------+ +--------------+ >+ | | >+ | | >+ | | >+ | | >+ | | >+ | | >+ 172.168.1.1/24 | >+ +--------------+ | >+ | br-phy1 | 172.168.1.2/24 >+ +--------------+ +---------------+ >+ | dpdk0/eth1 |==========================| eth1 | >+ +--------------+ +---------------+ >+ Host A with OVS. Remote host. >+ >+#### Prerequisites >+The host must be pre-installed with OVS-DPDK, QEMU and virtual machine >images. >+Please refer the installation guide of relevant modules for these >instructions. >+ovs-vswitchd and OVSDB server must be up and running before proceeding >to any >+of the configuration steps. >+ >+This setup guide covers only the required steps for setting up VxLAN >userspace >+tunneling. The same approach can be used for any other tunneling >protocols, by >+specifying the appropriate tunneling protocol type. >+ >+Note:- This configuration guide is for setting up the VxLAN tunneling on >one >+host(local host). The same steps have to perform on the remote host as >well for >+setting up a VM<->VM VxLAN tunnel setup. The only difference for setting >up the >+userspace-tunneling in remote node is , VM and VxLAN tunnel ip addresses >are >+different. >+ >+#### Configuration steps >+1. Create the ³br-int² bridge as below, >+ >+ ***HOST-A$ sudo ovs-vsctl --may-exist add-br br-int -- set Bridge >br-int datapath_type=netdev -- br-set-external-id br-int bridge-id >br-int -- set bridge br-int fail-mode=standalone*** >+ >+2. Add the VM interface to the ³br-int². >+ >+ ***HOST-A$ sudo ovs-vsctl add-port br-int vm_port0 -- set Interface >vm_port0 type=dpdkvhostuser*** >+ >+ `[³vm_port0² is the vhost-user interface name for the VM0. This >interface name >+must be used while qemu starting the virtual machine.]` >+ >+3. Start the VM(VM-0) using vhost-user network backend. The vhost-user >+interface name is "vm_port0". >+ >+4. Set the Ip address 192.168.1.1 to the VM interface. Run the following >+command inside the VM to set the Ip address. >+ >+ ***VM-0$ sudo ip addr add 192.168.1.1/24 dev eth0*** >+ >+ `[³eth0² is the interface inside VM. its possible to set the ip >address using >+"ifconfig" command as "sudo ifconfig eth0 192.168.1.1/24". Please use >either >+one command to set the ip address.]` >+ >+5. Attach the VxLAN tunnel interface to the ³br-int³ bridge. >+ >+ ***HOST-A$ sudo ovs-vsctl add-port br-int vxlan0 -- set interface >vxlan0 type=vxlan options:remote_ip=172.168.1.2*** >+ >+ `[³172.168.1.2² is the remote tunnel end point address, On remote host >this >+will be 172.168.1.1.]` >+ >+6. Create the ³br-phy1² bridge for attaching the physical interface. >+ >+ ***HOST-A$ sudo ovs-vsctl --may-exist add-br br-phy1 -- set Bridge >br-phy1 datapath_type=netdev -- br-set-external-id br-phy1 bridge-id >br-phy1 -- set bridge br-phy1 fail-mode=standalone >other_config:hwaddr="<mac address of eth1 interface>"*** >+ >+ `[³eth1² is the physical interface on the host, Assign mac address of >³eth1² >+to the ³br-phy1².]` >+ >+7. The physical port "eth1" can operate either as a KNI(kernel network >+interface) or a DPDK interface. Depending on the operating mode, attach >"eth1" >+to the "br-phy1" bridge as follows. >+ >+ Use step-7 if "eth1" is KNI. Please follow Step :8 rather than this >step in >+case "eth1" is a DPDK Interface.This step will cause to loose the >connectivity >+through ³eth1² (refer configuration problems in >https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openvswitc >h_ovs_blob_master_FAQ.md&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtX >t-uEs&r=SmB5nZacmXNq0gKCC1s_Cw5yUNjxgD4v5kJqZ2uWLlE&m=LdO039X1zZ5EL5hDBSK- >X5UByLfxhThnfXUHoZaJp2M&s=8zuC3ssoCA-R5eXDWhdYSsNszPcmW0vmQFE21KE9vBg&e= >for more details) for a >+while. The connectivity can be restored by moving the IP address to the >+³br-phy1² internal interface. The following command-set will do that, >+ >+ ***HOST-A$ sudo ovs-vsctl --timeout 10 add-port br-phy1 eth1*** >+ >+ ***HOST-A$ sudo ip addr add 172.168.1.1/24 dev br-phy1*** >+ >+ ***HOST-A$ sudo ip link set br-phy1 up*** >+ >+ ***HOST-A$ sudo ip addr flush dev eth1 2>/dev/null*** >+ >+ ***HOST-A$ sudo ip link set eth1 up*** >+ >+ ***HOST-A$ sudo iptables F*** >+ >+8. Steps for attaching ³eth1² to ³br-phy1² is slightly different in case >+³eth1² is a DPDK interface. DPDK interfaces are not managed by the >kernel, so >+the port details are not visible on any ³ip² commands. >+ >+ ***HOST-A$ sudo ./tools/dpdk_nic_bind.py --bind=igb_uio eth1*** >+ >+ `["dpdk_nic_bind.py" is a utility script to bind/unbind interfaces to >DPDK/Linux >+kernel.More details on bind/unbind can be found at >https://urldefense.proofpoint.com/v2/url?u=http-3A__dpdk.org_doc_guides_li >nux-5Fgsg_build-5Fdpdk.html-23binding-2Dand-2Dunbinding-2Dnetwork-2Dports- >2Dto-2Dfrom-2Dthe-2Dkernel-2Dmodules&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXV >eAw-YihVMNtXt-uEs&r=SmB5nZacmXNq0gKCC1s_Cw5yUNjxgD4v5kJqZ2uWLlE&m=LdO039X1 >zZ5EL5hDBSK-X5UByLfxhThnfXUHoZaJp2M&s=WAahc2Mv28gVX8eCe913qdXbU_pYWFi_cjzc >PBxOgLc&e= ]` >+ >+ ***HOST-A$ sudo ovs-vsctl --timeout 10 add-port br-phy1 dpdk0 -- set >Interface dpdk0 type=dpdk*** >+ >+ ***HOST-A$ sudo ip addr add 172.168.1.1/24 dev br-phy1*** >+ >+ ***HOST-A$ sudo ip link set br-phy1 up*** >+ >+ ***HOST-A$ sudo iptables F*** >+ >+ `[³eth1² is mapped to DPDK port ³dpdk0². DPDK driver assign port names >that >+starts with ³dpdk²]` >+ >+Now the traffic from ³VM0² will be VxLAN encapsulated and send out over >eth1/dpdk0 >+interface. >+ >+TCPDUMP doesnt work on DPDK interfaces(eth0) as its no longer managed by >the kernel. >+ >+#### Debugging >+ >+* Verify the created tunnel port details. >+ >+ ***HOST-A$ sudo ovs-appctl tnl/ports/show*** >+ >+* As mentioned before, its necessary that the vswitch should have the >arp-table >+entries to do tunnelling at userspace. The learned arp entries in the >vswitch >+can be verified by, >+ >+ ***HOST-A$ sudo ovs-appctl tnl/arp/show*** >+ >+ the arp entries can be flushed using, >+ >+ ***HOST-A$ sudo ovs-appctl tnl/arp/flush*** >+ >+ To set a specific arp entry, >+ >+ ***HOST-A$ sudo ovs-appctl tnl/arp/set <bridge> <ip addr> ><mac addr>*** >+ >+ In the above test setup, the following arp entries has to set in case >they are >+ not present. >+ >+ ***HOST-A$ sudo ovs-appctl tnl/arp/set br-int 172.168.1.1 <mac addr >of br-phy1>*** >+ >+ ***HOST-A$ sudo ovs-appctl tnl/arp/set br-phy1 172.168.1.2 <mac >addr of remote TEP>*** >+ >+* Similarly OVS uses routing table entries to xmit the tunnel packets. >The >+vswitch routing entries can be verified by, >+ >+ ***HOST-A$ sudo ovs-appctl ovs/route/show*** >+ >+ To delete the route entries. >+ >+ ***HOST-A$ sudo ovs-appctl ovs/route/del*** >+ >+* To lookup a route entry for a destination please use, >+ >+ ***HOST-A$ ovs-appctl ovs/route/lookup*** >+ >+* In case the route entries are missing for tunnel packet forwarding, it >can be >+added using the following command, >+ >+ ***HOST-A$ sudo ovs-appctl ovs/route/add 172.168.1.1/24 br-phy1*** >+ >+ This command instructs OVS to route all traffic destined for >³172.168.1.2² to >+ bridge ³br-phy1² >+ >+* To verify the range of tunneling source ports, >+ >+ ***HOST-A$ sudo ovs-appctl tnl/egress_port_range*** >+ >+* To see the configured datapath ports, >+ >+ ***HOST-A$ sudo ovs-appctl dpif/show*** >+ >+* To verify the flows programmed on the OVS datapath, >+ >+ ***HOST-A$ sudo ovs-appctl dpctl/dump-flows*** >+ >+ [This command shows how OVS forwards the packets in datapath.] >-- >2.1.4 > >_______________________________________________ >dev mailing list >[email protected] >https://urldefense.proofpoint.com/v2/url?u=http-3A__openvswitch.org_mailma >n_listinfo_dev&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=Sm >B5nZacmXNq0gKCC1s_Cw5yUNjxgD4v5kJqZ2uWLlE&m=LdO039X1zZ5EL5hDBSK-X5UByLfxhT >hnfXUHoZaJp2M&s=8IMF4lSKT6LrB5cEJkINsDKtlW_2tedPcqV0OFXLhUY&e= _______________________________________________ dev mailing list [email protected] http://openvswitch.org/mailman/listinfo/dev
