Regards _Sugesh
From: Geza Gemes [mailto:geza.ge...@gmail.com] Sent: Tuesday, October 11, 2016 2:34 PM To: Chandran, Sugesh <sugesh.chand...@intel.com>; discuss@openvswitch.org Subject: Re: [ovs-discuss] Behavior of netdev (dpdk) bridges with non-dpdkvhostuser ports On 10/11/2016 10:45 AM, Chandran, Sugesh wrote: Hi Geza, Regards _Sugesh From: discuss [mailto:discuss-boun...@openvswitch.org] On Behalf Of Geza Gemes Sent: Sunday, October 9, 2016 5:30 PM To: discuss@openvswitch.org<mailto:discuss@openvswitch.org> Subject: [ovs-discuss] Behavior of netdev (dpdk) bridges with non-dpdkvhostuser ports Hi, I've created a libvirt/KVM VM with Ubuntu 16.04 for experimenting with OVS 2.6 and DPDK 16.07 (from ubuntu cloud archive, newton), it has a number of NICs, out of which I've kept one for a kernel (ens16) and one for a DPDK (dpdk0) datapath. The bridge setup without virtual NICs: #ovs-vsctl show ef015869-3f47-45d4-af20-644f75208a92 Bridge "dpdk-br0" Port "dpdk-br0" Interface "dpdk-br0" type: internal Port "dpdk0" Interface "dpdk0" type: dpdk Bridge "non-dpdk-br1" Port "ens16" Interface "ens16" Port "non-dpdk-br1" Interface "non-dpdk-br1" type: internal ovs_version: "2.6.0" I've created an lxc container (I'm using the lxc daily ppa) with the following config: # grep -v ^# /var/lib/lxc/ovsdpdkbr/config lxc.include = /usr/share/lxc/config/ubuntu.common.conf lxc.rootfs = /var/lib/lxc/ovsdpdkbr/rootfs lxc.rootfs.backend = dir lxc.utsname = ovsdpdkbr lxc.arch = amd64 lxc.network.type = veth lxc.network.link = dpdk-br0 lxc.network.flags = up lxc.network.hwaddr = 00:16:3e:0c:c3:69 lxc.network.veth.pair = veth-dpdk Started the lxc container and expected to have no connectivity as lxc is not aware of the bridge being netdev: # ovs-vsctl show ef015869-3f47-45d4-af20-644f75208a92 Bridge "dpdk-br0" Port "dpdk-br0" Interface "dpdk-br0" type: internal Port veth-dpdk Interface veth-dpdk Port "dpdk0" Interface "dpdk0" type: dpdk Bridge "non-dpdk-br1" Port "ens16" Interface "ens16" Port "non-dpdk-br1" Interface "non-dpdk-br1" type: internal ovs_version: "2.6.0" The strange thing is, that it has got an IP address over eth0: $ ifconfig eth0 eth0 Link encap:Ethernet HWaddr 00:16:3e:0c:c3:69 inet addr:192.168.122.215 Bcast:192.168.122.255 Mask:255.255.255.0 inet6 addr: fe80::216:3eff:fe0c:c369/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:17 errors:0 dropped:0 overruns:0 frame:0 TX packets:11 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:2768 (2.7 KB) TX bytes:1374 (1.3 KB) and is able to ping the host: ubuntu@ovsdpdkbr:~$ ping -c 1 192.168.122.1 PING 192.168.122.1 (192.168.122.1) 56(84) bytes of data. 64 bytes from 192.168.122.1<http://192.168.122.1>: icmp_seq=1 ttl=64 time=0.290 ms --- 192.168.122.1 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.290/0.290/0.290/0.000 ms And the host is also able to ping the container: $ ping -c 1 192.168.122.215 PING 192.168.122.215 (192.168.122.215) 56(84) bytes of data. 64 bytes from 192.168.122.215<http://192.168.122.215>: icmp_seq=1 ttl=64 time=0.265 ms --- 192.168.122.215 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.265/0.265/0.265/0.000 ms But while sshd listens in the container: root@ovsdpdkbr:~# netstat -tunap Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:22<http://0.0.0.0:22> 0.0.0.0:* LISTEN 179/sshd tcp6 0 0 :::22 :::* LISTEN 179/sshd udp 0 0 0.0.0.0:68<http://0.0.0.0:68> 0.0.0.0:* 147/dhclient I cannot connect to it from the host: ssh -v -v -v ubuntu@192.168.122.215<mailto:ubuntu@192.168.122.215> OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014 debug1: Reading configuration data /home/egzagme/.ssh/config debug1: /home/egzagme/.ssh/config line 1: Applying options for 192.168.*.* debug1: Reading configuration data /etc/ssh/ssh_config debug1: /etc/ssh/ssh_config line 19: Applying options for * debug2: ssh_connect: needpriv 0 debug1: Connecting to 192.168.122.215 [192.168.122.215] port 22. Looks like ICMP and UDP packets go through somehow, but TCP does not. [Sugesh] Did you configure any static rules or its just normal action in OVS? Can you please confirm what rules are being installed in the OVS datapath? Have you tried ssh in container with different port than 22? Also is there any iptable rules that present in the host? Could someone please explain the observed behavior? Thank you in advance! Cheers, Geza Hi Sugesh, The packets shall be subject to the normal action only: # ovs-ofctl show dpdk-br0 OFPT_FEATURES_REPLY (xid=0x2): dpid:00008e3a9f11d64e n_tables:254, n_buffers:256 capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan mod_dl_src mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src mod_tp_dst 1(dpdk0): addr:52:54:00:03:64:b1 config: 0 state: 0 current: 10GB-FD advertised: FIBER supported: 1GB-HD 1GB-FD 10GB-FD COPPER FIBER AUTO_PAUSE peer: 10MB-FD 100MB-HD 100MB-FD 10GB-FD COPPER speed: 10000 Mbps now, 10000 Mbps max LOCAL(dpdk-br0): addr:8e:3a:9f:11:d6:4e config: PORT_DOWN state: LINK_DOWN current: 10MB-FD COPPER speed: 10 Mbps now, 0 Mbps max OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0 I have no iptables set up on the host: # iptables -L Chain INPUT (policy ACCEPT) target prot opt source destination ACCEPT tcp -- anywhere anywhere tcp dpt:domain ACCEPT udp -- anywhere anywhere udp dpt:domain ACCEPT tcp -- anywhere anywhere tcp dpt:bootps ACCEPT udp -- anywhere anywhere udp dpt:bootps Chain FORWARD (policy ACCEPT) target prot opt source destination ACCEPT all -- anywhere anywhere ACCEPT all -- anywhere anywhere Chain OUTPUT (policy ACCEPT) target prot opt source destination On the other hand I have an other bridge, with kernel datapath and there connectivity of lxc containers works as expected. My question is how communication between an bridge with dpdk datapath and an lxc container not having a dpdkvhostuser port and not being backed by hugepages is supposed [Sugesh] The ovs-vswitchd main thread will taken care of the kernel interfaces for the packet forwarding. The PMD threads are not involved in this packet handling, so hugepages. I am not very familiar with lxc OVS network setup. Definitely OVS forwards packets based on the configured rules when they landed in either through kernel or DPDK managed interface. You could verify whats happening in the ovs-dpdk data path using following commands ovs-appctcl dpctl/show –s netdev@ovs-netdev (watch these to see the port stats) ovs-appctl dpctl/dump-flows netdev@ovs-netdev Please have a look at the route table and arp in host for more debugging. In the above setup, 192.168.122.1 is the ip address assigned to the kernel interface ens16? So any packet destined for that subnet will forward to that interface than OVS? Is this assumption correct? to work? Thank you! Cheers, Geza
_______________________________________________ discuss mailing list discuss@openvswitch.org http://openvswitch.org/mailman/listinfo/discuss
