BTW, We are using the latest driver for the NIC.
From the physical nic, it shows this firmware version which seems to be the latest for XL710 NIC ubuntu@dskl12:~$ ethtool -i eth10 driver: i40e version: 2.4.10 firmware-version: 6.01 0x80003554 1.1747.0 One of vf interfaces, it shows the latest vf driver too. ubuntu@dskl13:~$ ethtool -i eth20 driver: i40evf version: 3.5.13 firmware-version: N/A -----Original Message----- From: dev <dev-boun...@dpdk.org> On Behalf Of Song, Kee SangX Sent: Monday, October 22, 2018 2:00 PM To: Honnappa Nagarahalli <honnappa.nagaraha...@arm.com>; Richardson, Bruce <bruce.richard...@intel.com>; De Lara Guarch, Pablo <pablo.de.lara.gua...@intel.com> Cc: dev@dpdk.org; Wang, Yipeng1 <yipeng1.w...@intel.com>; gavin...@arm.com; dharmik.thak...@arm.com; n...@arm.com Subject: Re: [dpdk-dev] Issues configuring OVS-DPDK in openstack queens Hi DPDK developers, While preparing NDA training at Intel, we are blocked due to this issue. So we need your input, and we will appreciate your help as soon as possible. Thanks in advance. -----Original Message----- From: dev <dev-boun...@dpdk.org> On Behalf Of Manojawa Paritala To: Richardson, Bruce <bruce.richard...@intel.com>; De Lara Guarch, Pablo <pablo.de.lara.gua...@intel.com> Cc: dev@dpdk.org; Wang, Yipeng1 <yipeng1.w...@intel.com>; honnappa.nagaraha...@arm.com; gavin...@arm.com; dharmik.thak...@arm.com; n...@arm.com Subject: [dpdk-dev] Issues configuring OVS-DPDK in openstack queens Hello All, On a 3 node (one controller + 2 compute), we configured Openstack Queens using OSA with OVS. On all the nodes, we defined br-mgmt as linux bridge, br-tun as private network and br-flat as external. Installation was successful and we could create networks and instances on Openstack. Below are the versions of the OVS packages used on each node. Controller :- openstack-vswitch - 2.9.0 Computes :- openstack-vswitch-dpdk - 2.9.0 (as we wanted to configure dpdk on the compute hosts) The openstack-vswitch-dpdk 2.9.0 package that we installed had dpdk version 17.11.3. When we tried to enable DPDK it failed with the below error. dpdk|ERR|DPDK not supported in this copy of Open vSwitch So, we downloaded the sources for dpdk 17.11.4 and openvswitch 2.9.2, built openvswitch with dpdk as suggested in the below official link. No issues on Openstack or OVS. http://docs.openvswitch.org/en/latest/intro/install/dpdk/ Then, we added the below parameters to OVS and everything looked ok. No issues in Openstack or OVS. $ovs-vsctl get Open_vSwitch . other_config {dpdk-extra="-n 2", dpdk-init="true", dpdk-lcore-mask="0x300000000000", dpdk-socket-mem="4096,4096", pmd-cpu-mask="0xf00003c0000", vhost-iommu-support="true"} Then on the compute node, in openvswitch_agent.ini file - OVS section, I added the below (based on the link https://docs.openstack.org/neutron/pike/contributor/internals/ovs_vhostuser.html ) and restarted neutron-openmvswitch-agent service. datapath_type=netdev vhostuser_socket_dir=/var/run/openvswitch After the above change, bridge br-flat is getting deleted from OVS. Attached are the logs after I restart the neutron-openmvswitch-agent service on the compute now. Not sure what the issue is. Can any of you please let me know if we are missing anything? Best Regards, PVMJ -------------- next part -------------- 2018-10-22T13:55:28.531Z|00338|rconn|INFO|br-flat1<->tcp:127.0.0.1:6633: connection closed by peer 2018-10-22T13:55:28.532Z|00339|rconn|INFO|br-vxlan<->tcp:127.0.0.1:6633: connection closed by peer 2018-10-22T13:55:28.532Z|00340|rconn|INFO|br-vlan<->tcp:127.0.0.1:6633: connection closed by peer 2018-10-22T13:55:28.532Z|00341|rconn|INFO|br-int<->tcp:127.0.0.1:6633: connection closed by peer 2018-10-22T13:55:28.532Z|00342|rconn|INFO|br-tun<->tcp:127.0.0.1:6633: connection closed by peer 2018-10-22T13:55:28.823Z|00343|rconn|INFO|br-flat1<->tcp:127.0.0.1:6633: connecting... 2018-10-22T13:55:28.823Z|00344|rconn|WARN|br-flat1<->tcp:127.0.0.1:6633: connection failed (Connection refused) 2018-10-22T13:55:28.823Z|00345|rconn|INFO|br-flat1<->tcp:127.0.0.1:6633: waiting 2 seconds before reconnect 2018-10-22T13:55:28.824Z|00346|rconn|INFO|br-vxlan<->tcp:127.0.0.1:6633: connecting... 2018-10-22T13:55:28.824Z|00347|rconn|WARN|br-vxlan<->tcp:127.0.0.1:6633: connection failed (Connection refused) 2018-10-22T13:55:28.824Z|00348|rconn|INFO|br-vxlan<->tcp:127.0.0.1:6633: waiting 2 seconds before reconnect 2018-10-22T13:55:28.824Z|00349|rconn|INFO|br-vlan<->tcp:127.0.0.1:6633: connecting... 2018-10-22T13:55:28.824Z|00350|rconn|WARN|br-vlan<->tcp:127.0.0.1:6633: connection failed (Connection refused) 2018-10-22T13:55:28.824Z|00351|rconn|INFO|br-vlan<->tcp:127.0.0.1:6633: waiting 2 seconds before reconnect 2018-10-22T13:55:28.824Z|00352|rconn|INFO|br-int<->tcp:127.0.0.1:6633: connecting... 2018-10-22T13:55:28.824Z|00353|rconn|WARN|br-int<->tcp:127.0.0.1:6633: connection failed (Connection refused) 2018-10-22T13:55:28.824Z|00354|rconn|INFO|br-int<->tcp:127.0.0.1:6633: waiting 2 seconds before reconnect 2018-10-22T13:55:28.824Z|00355|rconn|INFO|br-tun<->tcp:127.0.0.1:6633: connecting... 2018-10-22T13:55:28.824Z|00356|rconn|WARN|br-tun<->tcp:127.0.0.1:6633: connection failed (Connection refused) 2018-10-22T13:55:28.824Z|00357|rconn|INFO|br-tun<->tcp:127.0.0.1:6633: waiting 2 seconds before reconnect 2018-10-22T13:55:30.823Z|00358|rconn|INFO|br-flat1<->tcp:127.0.0.1:6633: connecting... 2018-10-22T13:55:30.823Z|00359|rconn|INFO|br-vxlan<->tcp:127.0.0.1:6633: connecting... 2018-10-22T13:55:30.823Z|00360|rconn|INFO|br-vlan<->tcp:127.0.0.1:6633: connecting... 2018-10-22T13:55:30.824Z|00361|rconn|INFO|br-int<->tcp:127.0.0.1:6633: connecting... 2018-10-22T13:55:30.824Z|00362|rconn|INFO|br-tun<->tcp:127.0.0.1:6633: connecting... 2018-10-22T13:55:30.870Z|00363|rconn|INFO|br-flat1<->tcp:127.0.0.1:6633: connected 2018-10-22T13:55:30.870Z|00364|rconn|INFO|br-vxlan<->tcp:127.0.0.1:6633: connected 2018-10-22T13:55:30.870Z|00365|rconn|INFO|br-vlan<->tcp:127.0.0.1:6633: connected 2018-10-22T13:55:30.871Z|00366|rconn|INFO|br-int<->tcp:127.0.0.1:6633: connected 2018-10-22T13:55:30.871Z|00367|rconn|INFO|br-tun<->tcp:127.0.0.1:6633: connected 2018-10-22T13:55:30.939Z|00368|bridge|INFO|bridge br-int: deleted interface int-br-vlan on port 2 2018-10-22T13:55:30.940Z|00369|bridge|INFO|bridge br-int: deleted interface br-int on port 65534 2018-10-22T13:55:30.940Z|00370|bridge|INFO|bridge br-int: deleted interface patch-tun on port 4 2018-10-22T13:55:30.940Z|00371|bridge|INFO|bridge br-int: deleted interface int-br-flat1 on port 1 2018-10-22T13:55:30.940Z|00372|bridge|INFO|bridge br-int: deleted interface int-br-vxlan on port 3 2018-10-22T13:55:31.530Z|00373|bridge|INFO|bridge br-int: added interface int-br-vlan on port 2 2018-10-22T13:55:31.535Z|00374|dpif_netdev|INFO|Core 42 on numa node 1 assigned port 'dpdk-p0' rx queue 0 (measured processing cycles 103120). 2018-10-22T13:55:31.535Z|00375|dpif_netdev|INFO|Core 18 on numa node 0 assigned port 'vhost-user-1' rx queue 0 (measured processing cycles 0). 2018-10-22T13:55:31.535Z|00376|bridge|INFO|bridge br-int: added interface br-int on port 65534 2018-10-22T13:55:31.536Z|00377|bridge|INFO|bridge br-int: added interface int-br-vxlan on port 3 2018-10-22T13:55:31.536Z|00378|bridge|INFO|bridge br-int: added interface int-br-flat1 on port 1 2018-10-22T13:55:31.536Z|00379|bridge|INFO|bridge br-int: added interface patch-tun on port 4 2018-10-22T13:55:31.536Z|00380|bridge|INFO|bridge br-int: using datapath ID 00002eeeccfa0248 2018-10-22T13:55:31.536Z|00381|connmgr|INFO|br-int: added service controller "punix:/var/run/openvswitch/br-int.mgmt" 2018-10-22T13:55:31.536Z|00382|connmgr|INFO|br-int: added primary controller "tcp:127.0.0.1:6633" 2018-10-22T13:55:31.536Z|00383|rconn|INFO|br-int<->tcp:127.0.0.1:6633: connecting... 2018-10-22T13:55:31.810Z|00384|rconn|INFO|br-int<->tcp:127.0.0.1:6633: connected 2018-10-22T13:55:32.041Z|00385|bridge|INFO|bridge br-flat1: deleted interface br-flat1 on port 65534 2018-10-22T13:55:32.042Z|00386|bridge|INFO|bridge br-flat1: deleted interface eth8 on port 1 2018-10-22T13:55:32.042Z|00387|bridge|INFO|bridge br-flat1: deleted interface phy-br-flat1 on port 2 2018-10-22T13:55:32.570Z|00388|dpif_netdev|INFO|Core 42 on numa node 1 assigned port 'dpdk-p0' rx queue 0 (measured processing cycles 103120). 2018-10-22T13:55:32.570Z|00389|dpif_netdev|INFO|Core 18 on numa node 0 assigned port 'vhost-user-1' rx queue 0 (measured processing cycles 0). 2018-10-22T13:55:32.572Z|00390|bridge|INFO|bridge br-flat1: added interface eth8 on port 1 2018-10-22T13:55:32.577Z|00391|dpif_netdev|INFO|Core 42 on numa node 1 assigned port 'dpdk-p0' rx queue 0 (measured processing cycles 103120). 2018-10-22T13:55:32.577Z|00392|dpif_netdev|INFO|Core 18 on numa node 0 assigned port 'vhost-user-1' rx queue 0 (measured processing cycles 0). 2018-10-22T13:55:32.577Z|00393|bridge|INFO|bridge br-flat1: added interface br-flat1 on port 65534 2018-10-22T13:55:32.577Z|00394|bridge|INFO|bridge br-flat1: added interface phy-br-flat1 on port 2 2018-10-22T13:55:32.577Z|00395|bridge|INFO|bridge br-flat1: using datapath ID 00003cfdfebcf7a8 2018-10-22T13:55:32.577Z|00396|connmgr|INFO|br-flat1: added service controller "punix:/var/run/openvswitch/br-flat1.mgmt" 2018-10-22T13:55:32.577Z|00397|connmgr|INFO|br-flat1: added primary controller "tcp:127.0.0.1:6633" 2018-10-22T13:55:32.577Z|00398|rconn|INFO|br-flat1<->tcp:127.0.0.1:6633: connecting... 2018-10-22T13:55:32.826Z|00399|netdev_linux|WARN|error receiving Ethernet packet on eth8: Network is down 2018-10-22T13:55:32.826Z|00400|dpif_netdev|ERR|error receiving data from eth8: Network is down 2018-10-22T13:55:32.827Z|00401|rconn|INFO|br-flat1<->tcp:127.0.0.1:6633: connected 2018-10-22T13:55:32.879Z|00402|bridge|INFO|bridge br-vlan: deleted interface phy-br-vlan on port 2 2018-10-22T13:55:32.880Z|00403|bridge|INFO|bridge br-vlan: deleted interface br-vlan on port 65534 2018-10-22T13:55:32.880Z|00404|bridge|INFO|bridge br-vlan: deleted interface eth10 on port 1 2018-10-22T13:55:33.431Z|00405|bridge|INFO|bridge br-vlan: added interface phy-br-vlan on port 2 2018-10-22T13:55:33.436Z|00406|dpif_netdev|INFO|Core 42 on numa node 1 assigned port 'dpdk-p0' rx queue 0 (measured processing cycles 103120). 2018-10-22T13:55:33.436Z|00407|dpif_netdev|INFO|Core 18 on numa node 0 assigned port 'vhost-user-1' rx queue 0 (measured processing cycles 0). 2018-10-22T13:55:33.436Z|00408|bridge|INFO|bridge br-vlan: added interface br-vlan on port 65534 2018-10-22T13:55:33.437Z|00409|dpif_netdev|INFO|Core 42 on numa node 1 assigned port 'dpdk-p0' rx queue 0 (measured processing cycles 103120). 2018-10-22T13:55:33.437Z|00410|dpif_netdev|INFO|Core 18 on numa node 0 assigned port 'vhost-user-1' rx queue 0 (measured processing cycles 0). 2018-10-22T13:55:33.437Z|00411|bridge|INFO|bridge br-vlan: added interface eth10 on port 1 2018-10-22T13:55:33.437Z|00412|bridge|INFO|bridge br-vlan: using datapath ID 00003cfdfebcf548 2018-10-22T13:55:33.437Z|00413|connmgr|INFO|br-vlan: added service controller "punix:/var/run/openvswitch/br-vlan.mgmt" 2018-10-22T13:55:33.437Z|00414|connmgr|INFO|br-vlan: added primary controller "tcp:127.0.0.1:6633" 2018-10-22T13:55:33.437Z|00415|rconn|INFO|br-vlan<->tcp:127.0.0.1:6633: connecting... 2018-10-22T13:55:33.677Z|00416|netdev_linux|WARN|error receiving Ethernet packet on eth10: Network is down 2018-10-22T13:55:33.677Z|00417|dpif_netdev|ERR|error receiving data from eth10: Network is down 2018-10-22T13:55:33.679Z|00418|rconn|INFO|br-vlan<->tcp:127.0.0.1:6633: connected 2018-10-22T13:55:33.689Z|00419|bridge|INFO|bridge br-flat1: deleted interface eth8 on port 1 2018-10-22T13:55:33.689Z|00420|dpif_netdev|INFO|Core 42 on numa node 1 assigned port 'dpdk-p0' rx queue 0 (measured processing cycles 103120). 2018-10-22T13:55:33.689Z|00421|dpif_netdev|INFO|Core 18 on numa node 0 assigned port 'vhost-user-1' rx queue 0 (measured processing cycles 0). 2018-10-22T13:55:33.731Z|00422|bridge|INFO|bridge br-flat1: using datapath ID 000026940e0e334d 2018-10-22T13:55:33.731Z|00423|rconn|INFO|br-flat1<->tcp:127.0.0.1:6633: disconnecting 2018-10-22T13:55:33.735Z|00424|connmgr|INFO|br-flat1<->tcp:127.0.0.1:6633: 2 flow_mods in the last 0 s (2 adds) 2018-10-22T13:55:33.748Z|00425|bridge|INFO|bridge br-flat1: deleted interface br-flat1 on port 65534 2018-10-22T13:55:33.748Z|00426|bridge|INFO|bridge br-flat1: deleted interface phy-br-flat1 on port 2 2018-10-22T13:55:33.997Z|00427|dpif_netdev|INFO|Core 42 on numa node 1 assigned port 'dpdk-p0' rx queue 0 (measured processing cycles 87660). 2018-10-22T13:55:33.997Z|00428|dpif_netdev|INFO|Core 18 on numa node 0 assigned port 'vhost-user-1' rx queue 0 (measured processing cycles 0). 2018-10-22T13:55:34.266Z|00429|bridge|INFO|bridge br-vlan: deleted interface eth10 on port 1 2018-10-22T13:55:34.267Z|00430|dpif_netdev|INFO|Core 42 on numa node 1 assigned port 'dpdk-p0' rx queue 0 (measured processing cycles 87660). 2018-10-22T13:55:34.267Z|00431|dpif_netdev|INFO|Core 18 on numa node 0 assigned port 'vhost-user-1' rx queue 0 (measured processing cycles 0). 2018-10-22T13:55:34.307Z|00432|bridge|INFO|bridge br-vlan: using datapath ID 00006220d7fae649 2018-10-22T13:55:34.307Z|00433|rconn|INFO|br-vlan<->tcp:127.0.0.1:6633: disconnecting 2018-10-22T13:55:34.324Z|00434|bridge|INFO|bridge br-vlan: deleted interface phy-br-vlan on port 2 2018-10-22T13:55:34.324Z|00435|bridge|INFO|bridge br-vlan: deleted interface br-vlan on port 65534 2018-10-22T13:55:34.564Z|00436|dpif_netdev|INFO|Core 42 on numa node 1 assigned port 'dpdk-p0' rx queue 0 (measured processing cycles 87660). 2018-10-22T13:55:34.564Z|00437|dpif_netdev|INFO|Core 18 on numa node 0 assigned port 'vhost-user-1' rx queue 0 (measured processing cycles 0). 2018-10-22T13:55:34.874Z|00438|poll_loop|INFO|Dropped 38 log messages in last 2670 seconds (most recently, 2667 seconds ago) due to excessive rate 2018-10-22T13:55:34.874Z|00439|poll_loop|INFO|wakeup due to [POLLIN] on fd 594 (FIFO pipe:[828052]) at lib/ovs-rcu.c:229 (80% CPU usage) 2018-10-22T13:55:34.874Z|00440|poll_loop|INFO|wakeup due to [POLLIN] on fd 594 (FIFO pipe:[828052]) at lib/ovs-rcu.c:229 (80% CPU usage) 2018-10-22T13:55:34.875Z|00441|rconn|INFO|br-vxlan<->tcp:127.0.0.1:6633: connection closed by peer 2018-10-22T13:55:34.875Z|00442|rconn|INFO|br-int<->tcp:127.0.0.1:6633: connection closed by peer 2018-10-22T13:55:34.875Z|00443|connmgr|INFO|br-int<->tcp:127.0.0.1:6633: 5 flow_mods in the 1 s starting 3 s ago (5 adds) 2018-10-22T13:55:34.875Z|00444|rconn|INFO|br-tun<->tcp:127.0.0.1:6633: connection closed by peer 2018-10-22T13:55:34.875Z|00445|poll_loop|INFO|wakeup due to [POLLIN] on fd 10 (NETLINK_ROUTE<->NETLINK_ROUTE) at lib/netlink-socket.c:1331 (80% CPU usage) 2018-10-22T13:55:34.875Z|00446|poll_loop|INFO|wakeup due to [POLLIN] on fd 11 (<->/var/run/openvswitch/db.sock) at lib/stream-fd.c:157 (80% CPU usage) 2018-10-22T13:55:34.881Z|00447|bridge|INFO|bridge br-flat1: added interface br-flat1 on port 65534 2018-10-22T13:55:34.881Z|00448|bridge|INFO|bridge br-flat1: using datapath ID 00008688d47b5149 2018-10-22T13:55:34.881Z|00449|connmgr|INFO|br-flat1: added service controller "punix:/var/run/openvswitch/br-flat1.mgmt" 2018-10-22T13:55:35.134Z|00450|poll_loop|INFO|wakeup due to [POLLIN] on fd 594 (FIFO pipe:[828052]) at lib/ovs-rcu.c:229 (80% CPU usage) 2018-10-22T13:55:35.135Z|00451|poll_loop|INFO|wakeup due to [POLLIN] on fd 594 (FIFO pipe:[828052]) at lib/ovs-rcu.c:229 (80% CPU usage) 2018-10-22T13:55:35.140Z|00452|poll_loop|INFO|wakeup due to [POLLIN] on fd 10 (NETLINK_ROUTE<->NETLINK_ROUTE) at lib/netlink-socket.c:1331 (80% CPU usage) 2018-10-22T13:55:35.140Z|00453|poll_loop|INFO|wakeup due to [POLLIN] on fd 11 (<->/var/run/openvswitch/db.sock) at lib/stream-fd.c:157 (80% CPU usage) 2018-10-22T13:55:35.141Z|00454|poll_loop|INFO|wakeup due to [POLLIN] on fd 594 (FIFO pipe:[828052]) at lib/ovs-rcu.c:229 (80% CPU usage) 2018-10-22T13:55:35.142Z|00455|poll_loop|INFO|wakeup due to [POLLIN] on fd 594 (FIFO pipe:[828052]) at vswitchd/bridge.c:385 (80% CPU usage) 2018-10-22T13:55:35.177Z|00456|bridge|INFO|bridge br-vlan: added interface br-vlan on port 65534 2018-10-22T13:55:35.178Z|00457|bridge|INFO|bridge br-vlan: using datapath ID 00003a70bfb95a40 2018-10-22T13:55:35.178Z|00458|connmgr|INFO|br-vlan: added service controller "punix:/var/run/openvswitch/br-vlan.mgmt" 2018-10-22T13:55:35.433Z|00459|bridge|INFO|bridge br-flat1: added interface eth8 on port 1 2018-10-22T13:55:35.433Z|00460|bridge|INFO|bridge br-flat1: using datapath ID 00003cfdfebcf7a8 2018-10-22T13:55:35.797Z|00461|bridge|INFO|bridge br-vlan: added interface eth10 on port 1 2018-10-22T13:55:35.797Z|00462|bridge|INFO|bridge br-vlan: using datapath ID 00003cfdfebcf548 2018-10-22T13:55:35.835Z|00463|rconn|INFO|br-vxlan<->tcp:127.0.0.1:6633: connecting... 2018-10-22T13:55:35.835Z|00464|rconn|WARN|br-vxlan<->tcp:127.0.0.1:6633: connection failed (Connection refused) 2018-10-22T13:55:35.835Z|00465|rconn|INFO|br-vxlan<->tcp:127.0.0.1:6633: waiting 2 seconds before reconnect 2018-10-22T13:55:35.835Z|00466|rconn|INFO|br-int<->tcp:127.0.0.1:6633: connecting... 2018-10-22T13:55:35.835Z|00467|rconn|WARN|br-int<->tcp:127.0.0.1:6633: connection failed (Connection refused) 2018-10-22T13:55:35.835Z|00468|rconn|INFO|br-int<->tcp:127.0.0.1:6633: waiting 2 seconds before reconnect 2018-10-22T13:55:35.836Z|00469|rconn|INFO|br-tun<->tcp:127.0.0.1:6633: connecting... 2018-10-22T13:55:35.836Z|00470|rconn|WARN|br-tun<->tcp:127.0.0.1:6633: connection failed (Connection refused) 2018-10-22T13:55:35.836Z|00471|rconn|INFO|br-tun<->tcp:127.0.0.1:6633: waiting 2 seconds before reconnect 2018-10-22T13:55:36.673Z|00472|bridge|INFO|bridge br-flat1: deleted interface eth8 on port 1 2018-10-22T13:55:36.675Z|00473|bridge|INFO|bridge br-flat1: using datapath ID 00008688d47b5149 2018-10-22T13:55:36.768Z|00474|bridge|INFO|bridge br-flat1: deleted interface br-flat1 on port 65534 2018-10-22T13:55:37.448Z|00475|bridge|INFO|bridge br-vlan: deleted interface eth10 on port 1 2018-10-22T13:55:37.449Z|00476|bridge|INFO|bridge br-vlan: using datapath ID 00003a70bfb95a40 2018-10-22T13:55:37.463Z|00477|bridge|INFO|bridge br-vlan: deleted interface br-vlan on port 65534 2018-10-22T13:55:38.037Z|00478|rconn|INFO|br-vxlan<->tcp:127.0.0.1:6633: connecting... 2018-10-22T13:55:38.037Z|00479|rconn|WARN|br-vxlan<->tcp:127.0.0.1:6633: connection failed (Connection refused) 2018-10-22T13:55:38.037Z|00480|rconn|INFO|br-vxlan<->tcp:127.0.0.1:6633: waiting 4 seconds before reconnect 2018-10-22T13:55:38.038Z|00481|rconn|INFO|br-int<->tcp:127.0.0.1:6633: connecting... 2018-10-22T13:55:38.038Z|00482|rconn|WARN|br-int<->tcp:127.0.0.1:6633: connection failed (Connection refused) 2018-10-22T13:55:38.038Z|00483|rconn|INFO|br-int<->tcp:127.0.0.1:6633: waiting 4 seconds before reconnect 2018-10-22T13:55:38.038Z|00484|rconn|INFO|br-tun<->tcp:127.0.0.1:6633: connecting... 2018-10-22T13:55:38.038Z|00485|rconn|WARN|br-tun<->tcp:127.0.0.1:6633: connection failed (Connection refused) 2018-10-22T13:55:38.038Z|00486|rconn|INFO|br-tun<->tcp:127.0.0.1:6633: waiting 4 seconds before reconnect 2018-10-22T13:55:38.044Z|00487|bridge|INFO|bridge br-flat1: added interface br-flat1 on port 65534 2018-10-22T13:55:38.044Z|00488|bridge|INFO|bridge br-flat1: using datapath ID 00008ed18924b545 2018-10-22T13:55:38.044Z|00489|connmgr|INFO|br-flat1: added service controller "punix:/var/run/openvswitch/br-flat1.mgmt" 2018-10-22T13:55:38.351Z|00490|bridge|INFO|bridge br-vlan: added interface br-vlan on port 65534 2018-10-22T13:55:38.352Z|00491|bridge|INFO|bridge br-vlan: using datapath ID 0000627270998843 2018-10-22T13:55:38.352Z|00492|connmgr|INFO|br-vlan: added service controller "punix:/var/run/openvswitch/br-vlan.mgmt" 2018-10-22T13:55:38.616Z|00493|bridge|INFO|bridge br-flat1: added interface eth8 on port 1 2018-10-22T13:55:38.616Z|00494|bridge|INFO|bridge br-flat1: using datapath ID 00003cfdfebcf7a8 2018-10-22T13:55:38.916Z|00495|bridge|INFO|bridge br-vlan: added interface eth10 on port 1 2018-10-22T13:55:38.916Z|00496|bridge|INFO|bridge br-vlan: using datapath ID 00003cfdfebcf548 2018-10-22T13:55:39.889Z|00497|bridge|INFO|bridge br-flat1: deleted interface eth8 on port 1 2018-10-22T13:55:39.890Z|00498|bridge|INFO|bridge br-flat1: using datapath ID 00008ed18924b545 2018-10-22T13:55:39.910Z|00499|bridge|INFO|bridge br-flat1: deleted interface br-flat1 on port 65534 2018-10-22T13:55:40.514Z|00500|bridge|INFO|bridge br-vlan: deleted interface eth10 on port 1 2018-10-22T13:55:40.515Z|00501|bridge|INFO|bridge br-vlan: using datapath ID 0000627270998843 2018-10-22T13:55:40.528Z|00502|bridge|INFO|bridge br-vlan: deleted interface br-vlan on port 65534 2018-10-22T13:55:41.052Z|00503|poll_loop|INFO|Dropped 630 log messages in last 6 seconds (most recently, 1 seconds ago) due to excessive rate 2018-10-22T13:55:41.053Z|00504|poll_loop|INFO|wakeup due to [POLLIN] on fd 594 (FIFO pipe:[828052]) at lib/ovs-rcu.c:229 (51% CPU usage) 2018-10-22T13:55:41.823Z|00505|rconn|INFO|br-vxlan<->tcp:127.0.0.1:6633: connecting... 2018-10-22T13:55:41.823Z|00506|rconn|INFO|br-int<->tcp:127.0.0.1:6633: connecting... 2018-10-22T13:55:41.823Z|00507|rconn|INFO|br-tun<->tcp:127.0.0.1:6633: connecting... 2018-10-22T13:55:41.827Z|00508|rconn|INFO|br-vxlan<->tcp:127.0.0.1:6633: connected 2018-10-22T13:55:41.827Z|00509|rconn|INFO|br-int<->tcp:127.0.0.1:6633: connected 2018-10-22T13:55:41.827Z|00510|rconn|INFO|br-tun<->tcp:127.0.0.1:6633: connected 2018-10-22T13:55:42.450Z|00511|rconn|INFO|br-vxlan<->tcp:127.0.0.1:6633: connection closed by peer 2018-10-22T13:55:42.450Z|00512|rconn|INFO|br-vxlan<->tcp:127.0.0.1:6633: continuing to retry connections in the background but suppressing further logging 2018-10-22T13:55:42.450Z|00513|rconn|INFO|br-int<->tcp:127.0.0.1:6633: connection closed by peer 2018-10-22T13:55:42.450Z|00514|rconn|INFO|br-int<->tcp:127.0.0.1:6633: continuing to retry connections in the background but suppressing further logging 2018-10-22T13:55:42.450Z|00515|connmgr|INFO|br-int<->tcp:127.0.0.1:6633: 4 flow_mods in the last 0 s (4 adds) 2018-10-22T13:55:42.450Z|00516|rconn|INFO|br-tun<->tcp:127.0.0.1:6633: connection closed by peer 2018-10-22T13:55:42.450Z|00517|rconn|INFO|br-tun<->tcp:127.0.0.1:6633: continuing to retry connections in the background but suppressing further logging 2018-10-22T13:55:49.827Z|00518|rconn|INFO|br-vxlan<->tcp:127.0.0.1:6633: connected 2018-10-22T13:55:49.828Z|00519|rconn|INFO|br-int<->tcp:127.0.0.1:6633: connected 2018-10-22T13:55:49.828Z|00520|rconn|INFO|br-tun<->tcp:127.0.0.1:6633: connected 2018-10-22T13:55:50.467Z|00521|rconn|INFO|br-vxlan<->tcp:127.0.0.1:6633: connection closed by peer 2018-10-22T13:55:50.467Z|00522|rconn|INFO|br-int<->tcp:127.0.0.1:6633: connection closed by peer 2018-10-22T13:55:50.467Z|00523|connmgr|INFO|br-int<->tcp:127.0.0.1:6633: 4 flow_mods in the last 0 s (4 adds) 2018-10-22T13:55:50.467Z|00524|rconn|INFO|br-tun<->tcp:127.0.0.1:6633: connection closed by peer
