On 10/11/2016 04:35 PM, Chandran, Sugesh wrote:
/Regards/
/_Sugesh/
*From:*Geza Gemes [mailto:geza.ge...@gmail.com]
*Sent:* Tuesday, October 11, 2016 2:34 PM
*To:* Chandran, Sugesh <sugesh.chand...@intel.com>;
discuss@openvswitch.org
*Subject:* Re: [ovs-discuss] Behavior of netdev (dpdk) bridges with
non-dpdkvhostuser ports
On 10/11/2016 10:45 AM, Chandran, Sugesh wrote:
Hi Geza,
/Regards/
/_Sugesh/
*From:*discuss [mailto:discuss-boun...@openvswitch.org] *On Behalf
Of *Geza Gemes
*Sent:* Sunday, October 9, 2016 5:30 PM
*To:* discuss@openvswitch.org <mailto:discuss@openvswitch.org>
*Subject:* [ovs-discuss] Behavior of netdev (dpdk) bridges with
non-dpdkvhostuser ports
Hi,
I've created a libvirt/KVM VM with Ubuntu 16.04 for experimenting
with OVS 2.6 and DPDK 16.07 (from ubuntu cloud archive, newton),
it has a number of NICs, out of which I've kept one for a kernel
(ens16) and one for a DPDK (dpdk0) datapath.
The bridge setup without virtual NICs:
#ovs-vsctl show
ef015869-3f47-45d4-af20-644f75208a92
Bridge "dpdk-br0"
Port "dpdk-br0"
Interface "dpdk-br0"
type: internal
Port "dpdk0"
Interface "dpdk0"
type: dpdk
Bridge "non-dpdk-br1"
Port "ens16"
Interface "ens16"
Port "non-dpdk-br1"
Interface "non-dpdk-br1"
type: internal
ovs_version: "2.6.0"
I've created an lxc container (I'm using the lxc daily ppa) with
the following config:
# grep -v ^# /var/lib/lxc/ovsdpdkbr/config
lxc.include = /usr/share/lxc/config/ubuntu.common.conf
lxc.rootfs = /var/lib/lxc/ovsdpdkbr/rootfs
lxc.rootfs.backend = dir
lxc.utsname = ovsdpdkbr
lxc.arch = amd64
lxc.network.type = veth
lxc.network.link = dpdk-br0
lxc.network.flags = up
lxc.network.hwaddr = 00:16:3e:0c:c3:69
lxc.network.veth.pair = veth-dpdk
Started the lxc container and expected to have no connectivity as
lxc is not aware of the bridge being netdev:
# ovs-vsctl show
ef015869-3f47-45d4-af20-644f75208a92
Bridge "dpdk-br0"
Port "dpdk-br0"
Interface "dpdk-br0"
type: internal
Port veth-dpdk
Interface veth-dpdk
Port "dpdk0"
Interface "dpdk0"
type: dpdk
Bridge "non-dpdk-br1"
Port "ens16"
Interface "ens16"
Port "non-dpdk-br1"
Interface "non-dpdk-br1"
type: internal
ovs_version: "2.6.0"
The strange thing is, that it has got an IP address over eth0:
$ ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:16:3e:0c:c3:69
inet addr:192.168.122.215 Bcast:192.168.122.255
Mask:255.255.255.0
inet6 addr: fe80::216:3eff:fe0c:c369/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:17 errors:0 dropped:0 overruns:0 frame:0
TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2768 (2.7 KB) TX bytes:1374 (1.3 KB)
and is able to ping the host:
ubuntu@ovsdpdkbr:~$ ping -c 1 192.168.122.1
PING 192.168.122.1 (192.168.122.1) 56(84) bytes of data.
64 bytes from 192.168.122.1 <http://192.168.122.1>: icmp_seq=1
ttl=64 time=0.290 ms
--- 192.168.122.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.290/0.290/0.290/0.000 ms
And the host is also able to ping the container:
$ ping -c 1 192.168.122.215
PING 192.168.122.215 (192.168.122.215) 56(84) bytes of data.
64 bytes from 192.168.122.215 <http://192.168.122.215>: icmp_seq=1
ttl=64 time=0.265 ms
--- 192.168.122.215 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.265/0.265/0.265/0.000 ms
But while sshd listens in the container:
root@ovsdpdkbr:~# netstat -tunap
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address
State PID/Program name
tcp 0 0 0.0.0.0:22 <http://0.0.0.0:22>
0.0.0.0:* LISTEN 179/sshd
tcp6 0 0 :::22 :::* LISTEN
179/sshd
udp 0 0 0.0.0.0:68 <http://0.0.0.0:68>
0.0.0.0:* 147/dhclient
I cannot connect to it from the host:
ssh -v -v -v ubuntu@192.168.122.215 <mailto:ubuntu@192.168.122.215>
OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014
debug1: Reading configuration data /home/egzagme/.ssh/config
debug1: /home/egzagme/.ssh/config line 1: Applying options for
192.168.*.*
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to 192.168.122.215 [192.168.122.215] port 22.
Looks like ICMP and UDP packets go through somehow, but TCP does not.
*/[Sugesh] Did you configure any static rules or its just normal
action in OVS?/*
*/Can you please confirm what rules are being installed in the OVS
datapath? /*
*/Have you tried ssh in container with different port than 22?/*
*/Also is there any iptable rules that present in the host?/*
*//*
Could someone please explain the observed behavior?
Thank you in advance!
Cheers,
Geza
Hi Sugesh,
The packets shall be subject to the normal action only:
# ovs-ofctl show dpdk-br0
OFPT_FEATURES_REPLY (xid=0x2): dpid:00008e3a9f11d64e
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan
mod_dl_src mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src
mod_tp_dst
1(dpdk0): addr:52:54:00:03:64:b1
config: 0
state: 0
current: 10GB-FD
advertised: FIBER
supported: 1GB-HD 1GB-FD 10GB-FD COPPER FIBER AUTO_PAUSE
peer: 10MB-FD 100MB-HD 100MB-FD 10GB-FD COPPER
speed: 10000 Mbps now, 10000 Mbps max
LOCAL(dpdk-br0): addr:8e:3a:9f:11:d6:4e
config: PORT_DOWN
state: LINK_DOWN
current: 10MB-FD COPPER
speed: 10 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0
I have no iptables set up on the host:
# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT tcp -- anywhere anywhere tcp dpt:domain
ACCEPT udp -- anywhere anywhere udp dpt:domain
ACCEPT tcp -- anywhere anywhere tcp dpt:bootps
ACCEPT udp -- anywhere anywhere udp dpt:bootps
Chain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
On the other hand I have an other bridge, with kernel datapath and
there connectivity of lxc containers works as expected. My question is
how communication between an bridge with dpdk datapath and an lxc
container not having a dpdkvhostuser port and not being backed by
hugepages is supposed
*/[Sugesh] The ovs-vswitchd main thread will taken care of the kernel
interfaces for the packet forwarding. The PMD threads are not involved
in this packet handling, so hugepages./*
*/I am not very familiar with lxc OVS network setup. Definitely OVS
forwards packets based on the configured rules when they landed in
either through kernel or DPDK managed interface. /*
*/You could verify whats happening in the ovs-dpdk data path using
following commands/*
*//*
*/ovs-appctcl dpctl/show –s netdev@ovs-netdev (watch these to see the
port stats)/*
*/ovs-appctl dpctl/dump-flows netdev@ovs-netdev/*
*//*
*/Please have a look at the route table and arp in host for more
debugging./*
*/In the above setup, /*192.168.122.1 */is the ip address assigned to
the kernel interface ens16? So any packet destined for that subnet
will forward to that interface than OVS? Is this assumption correct?/*
*//*
to work?
Thank you!
Cheers,
Geza
Hi,
My network setup looks like:
Where the host network with the dhcp server listening at 192.168.122.1
has 192.168.122.0/24
Ovs command:
# ovs-appctl dpctl/show –s netdev@ovs-netdev
ovs-vswitchd: opening datapath –s failed (No such device)
netdev@ovs-netdev:
lookups: hit:376 missed:21 lost:0
flows: 4
port 0: ovs-netdev (tap)
port 1: dpdk0 (dpdk: configured_rx_queues=1,
configured_tx_queues=1, mtu=1500, requested_rx_queues=1,
requested_tx_queues=3)
port 2: dpdk-br0 (tap)
port 3: veth-dpdk
ovs-appctl: ovs-vswitchd: server returned an error
suggest some kind of error. Arp table in the VM contains the host. I've
also tried with ens16 removed from the kernel datapath bridge but with
no difference.
Tomorrow I'll retry the setup with a qemu VM instead of an lxc container.
Cheers,
Geza
_______________________________________________
discuss mailing list
discuss@openvswitch.org
http://openvswitch.org/mailman/listinfo/discuss
