Hi,
I have a testbed, including 100 hypervisors running ovn-controller,
ovsdb-server, and ovs-vswitchd. When VM instances are created and ports are
bounded to the ovsdb-server, the OVN chassis process on certain node terminates
unexpected after the hundreds of ports are created and bounded. Following are
the log messages after the processes terminate.
------------- ovs-vswitchd.log -------------
2016-05-26T06:33:25.955Z|00231|bridge|INFO|bridge br0: added interface patch-provnet-lswitch_5837dd_0uNkGW-to-br-int on port 1
2016-05-26T06:33:25.955Z|00232|bridge|INFO|bridge br-int: added interface patch-br-int-to-provnet-lswitch_5837dd_0uNkGW on port 202
2016-05-26T06:33:29.293Z|00233|connmgr|INFO|br-int<->unix: 4519 flow_mods in the 9 s starting 10 s ago (4502 adds, 1 deletes, 16 modifications)
2016-05-26T06:34:29.293Z|00234|connmgr|INFO|br-int<->unix: 214 flow_mods in the last 59 s (214 adds)
2016-05-26T06:35:29.293Z|00235|connmgr|INFO|br-int<->unix: 123 flow_mods in the last 59 s (123 adds)
2016-05-26T06:36:29.293Z|00236|connmgr|INFO|br-int<->unix: 103 flow_mods in the 57 s starting 59 s ago (103 adds)
2016-05-26T06:37:29.293Z|00237|connmgr|INFO|br-int<->unix: 90 flow_mods in the 58 s starting 59 s ago (90 adds)
2016-05-26T06:38:29.318Z|00238|connmgr|INFO|br-int<->unix: 86 flow_mods in the 57 s starting 58 s ago (86 adds)
2016-05-26T06:39:29.294Z|00239|connmgr|INFO|br-int<->unix: 80 flow_mods in the 55 s starting 59 s ago (80 adds)
2016-05-26T06:40:29.302Z|00240|connmgr|INFO|br-int<->unix: 30 flow_mods in the 22 s starting 59 s ago (30 adds)
2016-05-26T06:41:29.293Z|00241|connmgr|INFO|br-int<->unix: 9 flow_mods in the 23 s starting 26 s ago (1 adds, 8 modifications)
2016-05-26T06:41:32.510Z|00242|bridge|INFO|bridge br-int: added interface lport_5837dd_RXXlL4 on port 203
2016-05-26T06:41:34.868Z|00243|bridge|INFO|bridge br-int: added interface lport_5837dd_LKaZD4 on port 204
2016-05-26T06:41:45.510Z|00244|connmgr|INFO|br-int<->unix: 836 flow_mods in the last 14 s (832 adds, 1 deletes, 3 modifications)
2016-05-26T06:41:50.191Z|00245|bridge|INFO|bridge br0: added interface patch-provnet-lswitch_5837dd_3MO740-to-br-int on port 2
2016-05-26T06:41:50.191Z|00246|bridge|INFO|bridge br-int: added interface patch-br-int-to-provnet-lswitch_5837dd_3MO740 on port 205
2016-05-26T06:42:05.150Z|00247|rconn|WARN|br-int<->unix: connection dropped (Connection reset by peer)
2016-05-26T06:42:05.196Z|00248|connmgr|INFO|br-int<->unix: 587 flow_mods in the last 4 s (586 adds, 1 deletes)
------------------------------------------------
------------- ovn-controller.log -------------
2016-05-26T06:41:56.796Z|00683|timeval|WARN|Unreasonably long 3666ms poll interval (472ms user, 0ms system)
2016-05-26T06:41:56.796Z|00684|timeval|WARN|faults: 2 minor, 0 major
2016-05-26T06:41:56.796Z|00685|timeval|WARN|context switches: 0 voluntary, 113 involuntary
2016-05-26T06:42:00.862Z|00686|timeval|WARN|Unreasonably long 4066ms poll interval (472ms user, 0ms system)
2016-05-26T06:42:00.862Z|00687|timeval|WARN|context switches: 0 voluntary, 127 involuntary
2016-05-26T06:42:05.147Z|00688|timeval|WARN|Unreasonably long 4285ms poll interval (508ms user, 0ms system)
2016-05-26T06:42:05.147Z|00689|timeval|WARN|context switches: 0 voluntary, 121 involuntary
2016-05-26T06:42:05.149Z|00690|rconn|ERR|unix:/usr/local/var/run/openvswitch/br-int.mgmt: no response to inactivity probe after 5 seconds, disconnecting
------------------------------------------------
------------- ovsdb-server.log -------------
016-05-26T06:25:15.580Z|00001|vlog|INFO|opened log file /usr/local/var/log/openvswitch/ovsdb-server.log
2016-05-26T06:25:15.583Z|00002|ovsdb_server|INFO|ovsdb-server (Open vSwitch) 2.5.90
2016-05-26T06:25:25.585Z|00003|memory|INFO|2432 kB peak resident set size after 10.0 seconds
2016-05-26T06:25:25.585Z|00004|memory|INFO|cells:1408 json-caches:2 monitors:2 sessions:2
2016-05-26T06:33:25.311Z|00005|jsonrpc|WARN|unix: receive error: Connection reset by peer
2016-05-26T06:33:25.311Z|00006|reconnect|WARN|unix: connection dropped (Connection reset by peer)
2016-05-26T06:33:25.581Z|00007|jsonrpc|WARN|unix: receive error: Connection reset by peer
2016-05-26T06:33:25.582Z|00008|reconnect|WARN|unix: connection dropped (Connection reset by peer)
2016-05-26T06:33:25.895Z|00009|memory|INFO|peak resident set size grew 73% in last 480.3 seconds, from 2432 kB to 4204 kB
2016-05-26T06:33:25.895Z|00010|memory|INFO|cells:11432 json-caches:2 monitors:2 sessions:2
------------------------------------------------
The OVS binaryies are compiled from git
    commit 1a523ec4fb50be3c887e3cdd0ae8b7bf6b4187e6
    Date:   Fri May 20 11:49:59 2016 -0700
    
Could anyone advise how to debug this issue? Thanks in advance.
 
- Hui

_______________________________________________
discuss mailing list
discuss@openvswitch.org
http://openvswitch.org/mailman/listinfo/discuss

Reply via email to