On 26 November 2015 at 02:58, Mikhail Bagrov <mbag...@cloudlinux.com> wrote:
> Hi Guru, Thanks much. I've added the internal bridge ports, started the
> vtep-emulator, added a docker container on all of my 3 nodes.
> ====================
> [root@ovs-1 ovs]# ovs-vsctl show
> 63796e59-59dd-4f8d-9ad2-2721257e1b60
> Bridge "br0"
> Port "p0"
> Interface "p0"
> type: internal
> Port "p1"
> Interface "p1"
> type: internal
> Port "1ab7473b25934_l"
> Interface "1ab7473b25934_l"
> Port "0000-p0-p"
> Interface "0000-p0-p"
> type: patch
> options: {peer="0000-p0-l"}
> Port "br0"
> Interface "br0"
> type: internal
> Bridge vtep_bfd
> Port vtep_bfd
> Interface vtep_bfd
> type: internal
> Port "bfd10.66.66.35"
> Interface "bfd10.66.66.35"
> type: vxlan
> options: {remote_ip="10.66.66.35"}
> Port "bfd10.66.66.36"
> Interface "bfd10.66.66.36"
> type: vxlan
> options: {remote_ip="10.66.66.36"}
> Port "bfd10.66.66.37"
> Interface "bfd10.66.66.37"
> type: vxlan
> options: {remote_ip="10.66.66.37"}
> Bridge "vtep_ls1"
> Port "vx5"
> Interface "vx5"
> type: vxlan
> options: {key="0", remote_ip="10.66.66.37"}
> Port "vx4"
> Interface "vx4"
> type: vxlan
> options: {key="0", remote_ip="10.66.66.36"}
> Port "vx6"
> Interface "vx6"
> type: vxlan
> options: {key="0", remote_ip="10.66.66.35"}
> Port "vtep_ls1"
> Interface "vtep_ls1"
> type: internal
> Port "0000-p0-l"
> Interface "0000-p0-l"
> type: patch
> options: {peer="0000-p0-p"}
> ======================
> Interface "1ab7473b25934_l" is a docker eth1 port
> ======================
> also added container's MAC to VTEP-IP bindings
> ======================
> [root@ovs-1 ovs]# vtep-ctl list-remote-macs ls0
> ucast-mac-remote
> 0a:34:43:e1:5a:45 -> vxlan_over_ipv4/10.66.66.35
> 3e:88:e6:eb:71:0a -> vxlan_over_ipv4/10.66.66.36
> e6:bd:d8:2c:39:2d -> vxlan_over_ipv4/10.66.66.37
>
> mcast-mac-remote
> ======================
> When I try to ping between containers, it fails, but ovs-ofctl dump-flows
> increments
> ======================
> [root@ovs-1 ovs]# ovs-ofctl dump-flows br0
> NXST_FLOW reply (xid=0x4):
> cookie=0x0, duration=11227.914s, table=0, n_packets=8, n_bytes=648,
> idle_age=11183, in_port=1 actions=output:3
> cookie=0x0, duration=11227.912s, table=0, n_packets=0, n_bytes=0,
> idle_age=11227, in_port=3 actions=output:1
> [root@ovs-1 ovs]# ovs-ofctl dump-flows br0
> NXST_FLOW reply (xid=0x4):
> cookie=0x0, duration=11230.203s, table=0, n_packets=8, n_bytes=648,
> idle_age=11185, in_port=1 actions=output:3
> cookie=0x0, duration=11230.201s, table=0, n_packets=0, n_bytes=0,
> idle_age=11230, in_port=3 actions=output:1
> ======================
> [root@ovs-1 ovs]# ovs-ofctl dump-ports br0 1ab7473b25934_l
> OFPST_PORT reply (xid=0x4): 1 ports
> port 4: rx pkts=7223, bytes=303678, drop=0, errs=0, frame=0, over=0,
> crc=0
> tx pkts=8, bytes=648, drop=0, errs=0, coll=0
> [root@ovs-1 ovs]# ovs-ofctl dump-ports br0 1ab7473b25934_l
> OFPST_PORT reply (xid=0x4): 1 ports
> port 4: rx pkts=7226, bytes=303804, drop=0, errs=0, frame=0, over=0,
> crc=0
> tx pkts=8, bytes=648, drop=0, errs=0, coll=0
>
There is a discrepancy above. As you can see the openflow flows in br0 has
two ofports - 3 and 1. But your container interface has a ofport of '4'. So
your container traffic will go nowhere.
If I had to guess, you have not added your container interface as a port to
physical switch using vtep-ctl. If you actually added that, then you have
not restarted your ovs-vtep python daemon.
> ======================
> and tcpdump on docker bridge captures only ARP requests, but no answers.
> ======================
> [root@ovs-1 ovs]# tcpdump -i 1ab7473b25934_l
> tcpdump: WARNING: 1ab7473b25934_l: no IPv4 address assigned
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on 1ab7473b25934_l, link-type EN10MB (Ethernet), capture size
> 65535 bytes
> 10:35:28.282498 ARP, Request who-has 10.111.111.113 tell 10.111.111.111,
> length 28
> 10:35:29.285339 ARP, Request who-has 10.111.111.113 tell 10.111.111.111,
> length 28
> 10:35:30.287307 ARP, Request who-has 10.111.111.113 tell 10.111.111.111,
> length 28
> =======================
> Logs look fine except /var/log/openvswitch/ovs-vswitchd.log
> =======================
> 2015-11-26T10:35:30.781Z|00754|ofproto|WARN|vtep_bfd: cannot configure bfd
> on nonexistent port 65535
> 2015-11-26T10:35:30.781Z|00755|ofproto|WARN|vtep_bfd: cannot configure
> LLDP on nonexistent port 65535
> 2015-11-26T10:35:30.781Z|00756|ofproto|WARN|vtep_bfd: cannot configure bfd
> on nonexistent port 65535
> 2015-11-26T10:35:30.781Z|00757|ofproto|WARN|vtep_bfd: cannot configure
> LLDP on nonexistent port 65535
> 2015-11-26T10:36:11.109Z|00758|ofproto|WARN|Dropped 59 log messages in
> last 60 seconds (most recently, 5 seconds ago) due to excessive rate
> 2015-11-26T10:36:11.109Z|00759|ofproto|WARN|vtep_bfd: cannot get STP stats
> on nonexistent port 65535
> 2015-11-26T10:37:11.120Z|00760|ofproto|WARN|Dropped 35 log messages in
> last 60 seconds (most recently, 5 seconds ago) due to excessive rate
> 2015-11-26T10:37:11.120Z|00761|ofproto|WARN|vtep_bfd: cannot get STP stats
> on nonexistent port 65535
> =======================
> Looks like I've missed something....
>
> Thanks in advance.
>
> On Wed, Nov 25, 2015 at 8:04 PM, Guru Shetty <g...@ovn.org> wrote:
>
>>
>>
>> On 23 November 2015 at 09:11, Mikhail Bagrov <mbag...@cloudlinux.com>
>> wrote:
>>
>>> Hi. I'm trying to implement the VTEP Emulator from
>>> https://github.com/openvswitch/ovs/blob/master/vtep/README.ovs-vtep.md
>>> I've built OVS from source on Centos7, trying to interconnect docker
>>> containers between different and within a single docker node. Still no
>>> traffic flows between containers.
>>>
>>
>> Debugging a VTEP emulator is a step by step process. How familiar are you
>> with openflow flows? i.e. do you understand what 'ovs-ofctl dump-flows br0'
>> says? When you ping from your docker container do you see a particular flow
>> stats getting incremented ?
>>
>>
>>>
>>> I have:
>>> [root@ovs11 ovs]# vtep-ctl list-remote-macs ls0
>>> ucast-mac-remote
>>> 7a:2c:ba:c0:62:16 -> vxlan_over_ipv4/10.2.2.102
>>> c2:fd:9f:ea:3f:e3 -> vxlan_over_ipv4/10.2.2.101
>>> de:eb:3c:62:65:fc -> vxlan_over_ipv4/10.2.2.103
>>> a mac to ip mapping table populated on each host.
>>> [root@ovs11 ovs]# ovs-vsctl show
>>> 82434185-c196-45dd-bcf1-1260e33ceae3
>>> Bridge vtep_bfd
>>> Port "bfd10.2.2.101"
>>> Interface "bfd10.2.2.101"
>>> type: vxlan
>>> options: {remote_ip="10.2.2.101"}
>>> Port "bfd10.2.2.102"
>>> Interface "bfd10.2.2.102"
>>> type: vxlan
>>> options: {remote_ip="10.2.2.102"}
>>> Port "bfd10.2.2.103"
>>> Interface "bfd10.2.2.103"
>>> type: vxlan
>>> options: {remote_ip="10.2.2.103"}
>>> Port vtep_bfd
>>> Interface vtep_bfd
>>> type: internal
>>> Bridge "vtep_ls1"
>>> Port "vx2"
>>> Interface "vx2"
>>> type: vxlan
>>> options: {key="0", remote_ip="10.2.2.102"}
>>> Port "vx1"
>>> Interface "vx1"
>>> type: vxlan
>>> options: {key="0", remote_ip="10.2.2.101"}
>>> Port "0000-p0-l"
>>> Interface "0000-p0-l"
>>> type: patch
>>> options: {peer="0000-p0-p"}
>>> Port "vx3"
>>> Interface "vx3"
>>> type: vxlan
>>> options: {key="0", remote_ip="10.2.2.103"}
>>> Port "vtep_ls1"
>>> Interface "vtep_ls1"
>>> type: internal
>>> Bridge "br0"
>>> Port "p1"
>>> Interface "p1"
>>> Port "br0"
>>> Interface "br0"
>>> type: internal
>>> Port "0000-p0-p"
>>> Interface "0000-p0-p"
>>> type: patch
>>> options: {peer="0000-p0-l"}
>>> Port "1d53fed7f9284_l"
>>> Interface "1d53fed7f9284_l"
>>> Port "p0"
>>> Interface "p0"
>>>
>>> Port "1d53fed7f9284_l" is a docker container's port.
>>> Settings look fine, but still no traffic flows. Thanks in advance.
>>>
>>>
>>> --
>>> Regards, Mikhail Bagrov
>>> DevOps at Kuberdock
>>>
>>>
>>> _______________________________________________
>>> discuss mailing list
>>> discuss@openvswitch.org
>>> http://openvswitch.org/mailman/listinfo/discuss
>>>
>>>
>>
>
>
> --
> Regards, Mikhail Bagrov
> DevOps at Kuberdock
>
>
_______________________________________________
discuss mailing list
discuss@openvswitch.org
http://openvswitch.org/mailman/listinfo/discuss
