2016-10-13 19:28 GMT+02:00 Géza Gémes <geza.ge...@gmail.com>: > Hi, > > Sorry for cross-posting, but I feel this might be an interesting topic for > the dev list as well. > > I've recreated my setup with qemu VM instead of lxc container and the > situation is the same. > Summary of my setup: > Ovs compiled with dpdk (from ubuntu-cloud-archive/newton) deployed on a > libvirt VM having 3 network connections to a shared network: > [image: Szövegközi kép 3] > I've set up two bridges: dpdk-br0 with dpdk and non-dpdk-br1 with kernel > datapath each having an interface to the host provided network. For both > bridges only the normal action is present, no flows were defined. > I forgot to mention, that intentionally I didn't use dpdkvhostuser ports even with the dpdk datapath bridge as the problem I try to investigate is whether there can be traffic in this scenario.
> If the qemu VM is connected to the kernel datapath bridge all traffic > passes between the host and the internal VM. If it is connected to the dpdk > bridge, udp and icmp traffic goes through, but tcp traffic does not. > > Could you please explain this behavior? > > Thank you in advance! > > Cheers, > > Geza > > 2016-10-11 21:18 GMT+02:00 Geza Gemes <geza.ge...@gmail.com>: > >> On 10/11/2016 04:35 PM, Chandran, Sugesh wrote: >> >>> >>> /Regards/ >>> >>> /_Sugesh/ >>> >>> *From:*Geza Gemes [mailto:geza.ge...@gmail.com] >>> *Sent:* Tuesday, October 11, 2016 2:34 PM >>> *To:* Chandran, Sugesh <sugesh.chand...@intel.com>; >>> discuss@openvswitch.org >>> *Subject:* Re: [ovs-discuss] Behavior of netdev (dpdk) bridges with >>> non-dpdkvhostuser ports >>> >>> On 10/11/2016 10:45 AM, Chandran, Sugesh wrote: >>> >>> Hi Geza, >>> >>> /Regards/ >>> >>> /_Sugesh/ >>> >>> *From:*discuss [mailto:discuss-boun...@openvswitch.org] *On Behalf >>> Of *Geza Gemes >>> *Sent:* Sunday, October 9, 2016 5:30 PM >>> *To:* discuss@openvswitch.org <mailto:discuss@openvswitch.org> >>> *Subject:* [ovs-discuss] Behavior of netdev (dpdk) bridges with >>> >>> non-dpdkvhostuser ports >>> >>> Hi, >>> >>> I've created a libvirt/KVM VM with Ubuntu 16.04 for experimenting >>> with OVS 2.6 and DPDK 16.07 (from ubuntu cloud archive, newton), >>> it has a number of NICs, out of which I've kept one for a kernel >>> (ens16) and one for a DPDK (dpdk0) datapath. >>> >>> The bridge setup without virtual NICs: >>> >>> #ovs-vsctl show >>> >>> ef015869-3f47-45d4-af20-644f75208a92 >>> >>> Bridge "dpdk-br0" >>> >>> Port "dpdk-br0" >>> >>> Interface "dpdk-br0" >>> >>> type: internal >>> >>> Port "dpdk0" >>> >>> Interface "dpdk0" >>> >>> type: dpdk >>> >>> Bridge "non-dpdk-br1" >>> >>> Port "ens16" >>> >>> Interface "ens16" >>> >>> Port "non-dpdk-br1" >>> >>> Interface "non-dpdk-br1" >>> >>> type: internal >>> >>> ovs_version: "2.6.0" >>> >>> I've created an lxc container (I'm using the lxc daily ppa) with >>> the following config: >>> >>> # grep -v ^# /var/lib/lxc/ovsdpdkbr/config >>> >>> lxc.include = /usr/share/lxc/config/ubuntu.common.conf >>> >>> lxc.rootfs = /var/lib/lxc/ovsdpdkbr/rootfs >>> >>> lxc.rootfs.backend = dir >>> >>> lxc.utsname = ovsdpdkbr >>> >>> lxc.arch = amd64 >>> >>> lxc.network.type = veth >>> >>> lxc.network.link = dpdk-br0 >>> >>> lxc.network.flags = up >>> >>> lxc.network.hwaddr = 00:16:3e:0c:c3:69 >>> >>> lxc.network.veth.pair = veth-dpdk >>> >>> Started the lxc container and expected to have no connectivity as >>> lxc is not aware of the bridge being netdev: >>> >>> # ovs-vsctl show >>> >>> ef015869-3f47-45d4-af20-644f75208a92 >>> >>> Bridge "dpdk-br0" >>> >>> Port "dpdk-br0" >>> >>> Interface "dpdk-br0" >>> >>> type: internal >>> >>> Port veth-dpdk >>> >>> Interface veth-dpdk >>> >>> Port "dpdk0" >>> >>> Interface "dpdk0" >>> >>> type: dpdk >>> >>> Bridge "non-dpdk-br1" >>> >>> Port "ens16" >>> >>> Interface "ens16" >>> >>> Port "non-dpdk-br1" >>> >>> Interface "non-dpdk-br1" >>> >>> type: internal >>> >>> ovs_version: "2.6.0" >>> >>> The strange thing is, that it has got an IP address over eth0: >>> >>> $ ifconfig eth0 >>> >>> eth0 Link encap:Ethernet HWaddr 00:16:3e:0c:c3:69 >>> >>> inet addr:192.168.122.215 Bcast:192.168.122.255 >>> Mask:255.255.255.0 >>> >>> inet6 addr: fe80::216:3eff:fe0c:c369/64 Scope:Link >>> >>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>> >>> RX packets:17 errors:0 dropped:0 overruns:0 frame:0 >>> >>> TX packets:11 errors:0 dropped:0 overruns:0 carrier:0 >>> >>> collisions:0 txqueuelen:1000 >>> >>> RX bytes:2768 (2.7 KB) TX bytes:1374 (1.3 KB) >>> >>> and is able to ping the host: >>> >>> ubuntu@ovsdpdkbr:~$ ping -c 1 192.168.122.1 >>> >>> PING 192.168.122.1 (192.168.122.1) 56(84) bytes of data. >>> >>> 64 bytes from 192.168.122.1 <http://192.168.122.1>: icmp_seq=1 >>> ttl=64 time=0.290 ms >>> >>> --- 192.168.122.1 ping statistics --- >>> >>> 1 packets transmitted, 1 received, 0% packet loss, time 0ms >>> >>> rtt min/avg/max/mdev = 0.290/0.290/0.290/0.000 ms >>> >>> And the host is also able to ping the container: >>> >>> $ ping -c 1 192.168.122.215 >>> >>> PING 192.168.122.215 (192.168.122.215) 56(84) bytes of data. >>> >>> 64 bytes from 192.168.122.215 <http://192.168.122.215>: icmp_seq=1 >>> ttl=64 time=0.265 ms >>> >>> --- 192.168.122.215 ping statistics --- >>> >>> 1 packets transmitted, 1 received, 0% packet loss, time 0ms >>> >>> rtt min/avg/max/mdev = 0.265/0.265/0.265/0.000 ms >>> >>> But while sshd listens in the container: >>> >>> root@ovsdpdkbr:~# netstat -tunap >>> >>> Active Internet connections (servers and established) >>> >>> Proto Recv-Q Send-Q Local Address Foreign Address >>> State PID/Program name >>> >>> tcp 0 0 0.0.0.0:22 <http://0.0.0.0:22> >>> 0.0.0.0:* LISTEN 179/sshd >>> >>> tcp6 0 0 :::22 :::* LISTEN >>> 179/sshd >>> >>> udp 0 0 0.0.0.0:68 <http://0.0.0.0:68> >>> 0.0.0.0:* 147/dhclient >>> >>> I cannot connect to it from the host: >>> >>> ssh -v -v -v ubuntu@192.168.122.215 <mailto:ubuntu@192.168.122.215> >>> >>> OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014 >>> >>> debug1: Reading configuration data /home/egzagme/.ssh/config >>> >>> debug1: /home/egzagme/.ssh/config line 1: Applying options for >>> 192.168.*.* >>> >>> debug1: Reading configuration data /etc/ssh/ssh_config >>> >>> debug1: /etc/ssh/ssh_config line 19: Applying options for * >>> >>> debug2: ssh_connect: needpriv 0 >>> >>> debug1: Connecting to 192.168.122.215 [192.168.122.215] port 22. >>> >>> Looks like ICMP and UDP packets go through somehow, but TCP does not. >>> >>> */[Sugesh] Did you configure any static rules or its just normal >>> action in OVS?/* >>> >>> */Can you please confirm what rules are being installed in the OVS >>> datapath? /* >>> >>> */Have you tried ssh in container with different port than 22?/* >>> >>> */Also is there any iptable rules that present in the host?/* >>> >>> *//* >>> >>> >>> Could someone please explain the observed behavior? >>> >>> Thank you in advance! >>> >>> Cheers, >>> >>> Geza >>> >>> Hi Sugesh, >>> >>> The packets shall be subject to the normal action only: >>> >>> # ovs-ofctl show dpdk-br0 >>> OFPT_FEATURES_REPLY (xid=0x2): dpid:00008e3a9f11d64e >>> n_tables:254, n_buffers:256 >>> capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP >>> actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan mod_dl_src >>> mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src mod_tp_dst >>> 1(dpdk0): addr:52:54:00:03:64:b1 >>> config: 0 >>> state: 0 >>> current: 10GB-FD >>> advertised: FIBER >>> supported: 1GB-HD 1GB-FD 10GB-FD COPPER FIBER AUTO_PAUSE >>> peer: 10MB-FD 100MB-HD 100MB-FD 10GB-FD COPPER >>> speed: 10000 Mbps now, 10000 Mbps max >>> LOCAL(dpdk-br0): addr:8e:3a:9f:11:d6:4e >>> config: PORT_DOWN >>> state: LINK_DOWN >>> current: 10MB-FD COPPER >>> speed: 10 Mbps now, 0 Mbps max >>> OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0 >>> >>> I have no iptables set up on the host: >>> >>> # iptables -L >>> Chain INPUT (policy ACCEPT) >>> target prot opt source destination >>> ACCEPT tcp -- anywhere anywhere tcp dpt:domain >>> ACCEPT udp -- anywhere anywhere udp dpt:domain >>> ACCEPT tcp -- anywhere anywhere tcp dpt:bootps >>> ACCEPT udp -- anywhere anywhere udp dpt:bootps >>> >>> Chain FORWARD (policy ACCEPT) >>> target prot opt source destination >>> ACCEPT all -- anywhere anywhere >>> ACCEPT all -- anywhere anywhere >>> >>> Chain OUTPUT (policy ACCEPT) >>> target prot opt source destination >>> >>> On the other hand I have an other bridge, with kernel datapath and there >>> connectivity of lxc containers works as expected. My question is how >>> communication between an bridge with dpdk datapath and an lxc container not >>> having a dpdkvhostuser port and not being backed by hugepages is supposed >>> >>> */[Sugesh] The ovs-vswitchd main thread will taken care of the kernel >>> interfaces for the packet forwarding. The PMD threads are not involved in >>> this packet handling, so hugepages./* >>> >>> */I am not very familiar with lxc OVS network setup. Definitely OVS >>> forwards packets based on the configured rules when they landed in either >>> through kernel or DPDK managed interface. /* >>> >>> */You could verify whats happening in the ovs-dpdk data path using >>> following commands/* >>> >>> *//* >>> >>> */ovs-appctcl dpctl/show –s netdev@ovs-netdev (watch these to see the >>> port stats)/* >>> >>> */ovs-appctl dpctl/dump-flows netdev@ovs-netdev/* >>> >>> *//* >>> >>> */Please have a look at the route table and arp in host for more >>> debugging./* >>> >>> */In the above setup, /*192.168.122.1 */is the ip address assigned to >>> the kernel interface ens16? So any packet destined for that subnet will >>> forward to that interface than OVS? Is this assumption correct?/* >>> >>> *//* >>> >>> to work? >>> >>> Thank you! >>> >>> Cheers, >>> >>> Geza >>> >>> Hi, >> >> My network setup looks like: >> >> >> Where the host network with the dhcp server listening at 192.168.122.1 >> has 192.168.122.0/24 >> >> Ovs command: >> >> # ovs-appctl dpctl/show –s netdev@ovs-netdev >> ovs-vswitchd: opening datapath –s failed (No such device) >> netdev@ovs-netdev: >> lookups: hit:376 missed:21 lost:0 >> flows: 4 >> port 0: ovs-netdev (tap) >> port 1: dpdk0 (dpdk: configured_rx_queues=1, configured_tx_queues=1, >> mtu=1500, requested_rx_queues=1, requested_tx_queues=3) >> port 2: dpdk-br0 (tap) >> port 3: veth-dpdk >> ovs-appctl: ovs-vswitchd: server returned an error >> >> suggest some kind of error. Arp table in the VM contains the host. I've >> also tried with ens16 removed from the kernel datapath bridge but with no >> difference. >> >> Tomorrow I'll retry the setup with a qemu VM instead of an lxc container. >> >> Cheers, >> >> Geza >> >> >> >> >
_______________________________________________ discuss mailing list discuss@openvswitch.org http://openvswitch.org/mailman/listinfo/discuss
