Public bug reported:
In kubernetes environment is very difficult to make sure that neutron-
openvswitch-agent pod starts after openvswitch db and vswitchd pods
during kubernetes cluster startup process. As a result it does not find
socket, fails and stay in failed state. If instead of just failing it
could retry several times then result would be different as by that time
openvswitch would have come up and created db socket. It would be great
to add retry mechanism to neutron-openvswitch-agent to make it more
robust in kubernetes environment.
neutron (9.0.0)
neutron-lib (0.4.0)
python-neutronclient (6.0.0)
2016-10-17 13:05:57.922 36 INFO ryu.base.app_manager [-] loading app
ryu.app.ofctl.service
2016-10-17 13:05:57.984 36 INFO ryu.base.app_manager [-] loading app
ryu.controller.ofp_handler
2016-10-17 13:05:57.989 36 INFO ryu.base.app_manager [-] instantiating app
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ovs_ryuapp of
OVSNeutronAgentRyuApp
2016-10-17 13:05:57.990 36 INFO ryu.base.app_manager [-] instantiating app
ryu.controller.ofp_handler of OFPHandler
2016-10-17 13:05:57.991 36 INFO ryu.base.app_manager [-] instantiating app
ryu.app.ofctl.service of OfctlService
2016-10-17 13:05:57.994 36 DEBUG neutron.callbacks.manager [-] Subscribe:
<function init_handler at 0x4549ed8> Open vSwitch agent after_init subscribe
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/callbacks/manager.py:42
2016-10-17 13:05:57.995 36 DEBUG neutron.agent.linux.utils [-] Running command:
['ip', 'addr', 'show', 'to', '172.29.75.24'] create_process
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/utils.py:83
2016-10-17 13:05:58.181 36 DEBUG neutron.agent.linux.utils [-] Exit code: 0
execute
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/utils.py:140
2016-10-17 13:05:58.225 36 DEBUG neutron.agent.linux.utils [-] Running command:
['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-vsctl',
'set-manager', 'ptcp:6640:0.0.0.0'] create_process
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/utils.py:83
2016-10-17 13:05:58.792 36 ERROR neutron.agent.linux.utils [-] Exit code: 1;
Stdin: ; Stdout: ; Stderr: ovs-vsctl: unix:/var/run/openvswitch/db.sock:
database connection failed (No such file or directory)
2016-10-17 13:05:58.793 36 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [-] Exit code:
1; Stdin: ; Stdout: ; Stderr: ovs-vsctl: unix:/var/run/openvswitch/db.sock:
database connection failed (No such file or directory)
Agent terminated!
2016-10-17 13:05:58.796 36 ERROR ryu.lib.hub [-] hub: uncaught exception:
Traceback (most recent call last):
File "/var/lib/kolla/venv/lib/python2.7/site-packages/ryu/lib/hub.py", line
54, in _launch
return func(*args, **kwargs)
File
"/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ovs_ryuapp.py",
line 37, in agent_main_wrapper
ovs_agent.main(bridge_classes)
File
"/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py",
line 2174, in main
sys.exit(1)
SystemExit: 1
** Affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1634123
Title:
neutron openvswitch agent exists if unix:/var/run/openvswitch/db.sock
is not yet created
Status in neutron:
New
Bug description:
In kubernetes environment is very difficult to make sure that neutron-
openvswitch-agent pod starts after openvswitch db and vswitchd pods
during kubernetes cluster startup process. As a result it does not
find socket, fails and stay in failed state. If instead of just
failing it could retry several times then result would be different as
by that time openvswitch would have come up and created db socket. It
would be great to add retry mechanism to neutron-openvswitch-agent to
make it more robust in kubernetes environment.
neutron (9.0.0)
neutron-lib (0.4.0)
python-neutronclient (6.0.0)
2016-10-17 13:05:57.922 36 INFO ryu.base.app_manager [-] loading app
ryu.app.ofctl.service
2016-10-17 13:05:57.984 36 INFO ryu.base.app_manager [-] loading app
ryu.controller.ofp_handler
2016-10-17 13:05:57.989 36 INFO ryu.base.app_manager [-] instantiating app
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ovs_ryuapp of
OVSNeutronAgentRyuApp
2016-10-17 13:05:57.990 36 INFO ryu.base.app_manager [-] instantiating app
ryu.controller.ofp_handler of OFPHandler
2016-10-17 13:05:57.991 36 INFO ryu.base.app_manager [-] instantiating app
ryu.app.ofctl.service of OfctlService
2016-10-17 13:05:57.994 36 DEBUG neutron.callbacks.manager [-] Subscribe:
<function init_handler at 0x4549ed8> Open vSwitch agent after_init subscribe
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/callbacks/manager.py:42
2016-10-17 13:05:57.995 36 DEBUG neutron.agent.linux.utils [-] Running
command: ['ip', 'addr', 'show', 'to', '172.29.75.24'] create_process
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/utils.py:83
2016-10-17 13:05:58.181 36 DEBUG neutron.agent.linux.utils [-] Exit code: 0
execute
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/utils.py:140
2016-10-17 13:05:58.225 36 DEBUG neutron.agent.linux.utils [-] Running
command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf',
'ovs-vsctl', 'set-manager', 'ptcp:6640:0.0.0.0'] create_process
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/utils.py:83
2016-10-17 13:05:58.792 36 ERROR neutron.agent.linux.utils [-] Exit code: 1;
Stdin: ; Stdout: ; Stderr: ovs-vsctl: unix:/var/run/openvswitch/db.sock:
database connection failed (No such file or directory)
2016-10-17 13:05:58.793 36 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [-] Exit code:
1; Stdin: ; Stdout: ; Stderr: ovs-vsctl: unix:/var/run/openvswitch/db.sock:
database connection failed (No such file or directory)
Agent terminated!
2016-10-17 13:05:58.796 36 ERROR ryu.lib.hub [-] hub: uncaught exception:
Traceback (most recent call last):
File "/var/lib/kolla/venv/lib/python2.7/site-packages/ryu/lib/hub.py", line
54, in _launch
return func(*args, **kwargs)
File
"/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ovs_ryuapp.py",
line 37, in agent_main_wrapper
ovs_agent.main(bridge_classes)
File
"/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py",
line 2174, in main
sys.exit(1)
SystemExit: 1
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1634123/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
