I patched this back in liberty. What version of Openstack are you using ? Saverio
2017-06-08 19:36 GMT+02:00 Grant Morley <gr...@absolutedevops.io>: > Ignore that now all, > > Managed to fix it by restarting the l3-agent. Looks like it must have been > cached in memory. > > Thanks, > > On 08/06/17 18:07, Grant Morley wrote: > > Hi All, > > We have noticed in our neutron-l3-agent logs that there are a number of > routers that neutron seems to think exist ( but they no longer exist in the > data base ) and it is constantly trying to delete them. However because > they don't appear to exist, so neutron seems to get stuck in a loop of > trying to delete routers that are no longer there. > > We originally noticed that neutron was trying to delete the > "qrouter-xxxx-xxx-xxxx" and that didn't exist. So we added the router > manually with "ip netns add qrouter-xxx-xxx-xxx-xxx" which then deletes the > router however we then get the following error messages: > > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent [-] Error while > deleting router da7b633a-233b-46d1-ba3d-315b3bee6a61 > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent Traceback (most > recent call last): > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File > "/openstack/venvs/neutron-13.3.8/lib/python2.7/site- > packages/neutron/agent/l3/agent.py", line 359, in _safe_router_removed > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent > self._router_removed(router_id) > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File > "/openstack/venvs/neutron-13.3.8/lib/python2.7/site- > packages/neutron/agent/l3/agent.py", line 377, in _router_removed > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent > ri.delete(self) > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File > "/openstack/venvs/neutron-13.3.8/lib/python2.7/site- > packages/neutron/agent/l3/ha_router.py", line 380, in delete > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent > super(HaRouter, self).delete(agent) > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File > "/openstack/venvs/neutron-13.3.8/lib/python2.7/site- > packages/neutron/agent/l3/router_info.py", line 361, in delete > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent > self.process_delete(agent) > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File > "/openstack/venvs/neutron-13.3.8/lib/python2.7/site- > packages/neutron/common/utils.py", line 385, in call > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent self.logger(e) > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File > "/openstack/venvs/neutron-13.3.8/lib/python2.7/site- > packages/oslo_utils/excutils.py", line 220, in __exit__ > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent > self.force_reraise() > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File > "/openstack/venvs/neutron-13.3.8/lib/python2.7/site- > packages/oslo_utils/excutils.py", line 196, in force_reraise > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent > six.reraise(self.type_, self.value, self.tb) > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File > "/openstack/venvs/neutron-13.3.8/lib/python2.7/site- > packages/neutron/common/utils.py", line 382, in call > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent return > func(*args, **kwargs) > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File > "/openstack/venvs/neutron-13.3.8/lib/python2.7/site- > packages/neutron/agent/l3/router_info.py", line 972, in process_delete > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent > self._process_external_on_delete(agent) > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File > "/openstack/venvs/neutron-13.3.8/lib/python2.7/site- > packages/neutron/agent/l3/router_info.py", line 794, in > _process_external_on_delete > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent > self._process_external_gateway(ex_gw_port, agent.pd) > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File > "/openstack/venvs/neutron-13.3.8/lib/python2.7/site- > packages/neutron/agent/l3/router_info.py", line 693, in > _process_external_gateway > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent > self.external_gateway_removed(self.ex_gw_port, interface_name) > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File > "/openstack/venvs/neutron-13.3.8/lib/python2.7/site- > packages/neutron/agent/l3/ha_router.py", line 371, in > external_gateway_removed > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent > interface_name) > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File > "/openstack/venvs/neutron-13.3.8/lib/python2.7/site- > packages/neutron/agent/l3/router_info.py", line 668, in > external_gateway_removed > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent > ip_addr['prefixlen'])) > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File > "/openstack/venvs/neutron-13.3.8/lib/python2.7/site- > packages/neutron/agent/l3/router_info.py", line 286, in > remove_external_gateway_ip > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent > device.delete_addr_and_conntrack_state(ip_cidr) > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File > "/openstack/venvs/neutron-13.3.8/lib/python2.7/site- > packages/neutron/agent/linux/ip_lib.py", line 290, in > delete_addr_and_conntrack_state > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent > self.addr.delete(cidr) > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File > "/openstack/venvs/neutron-13.3.8/lib/python2.7/site- > packages/neutron/agent/linux/ip_lib.py", line 580, in delete > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent 'dev', > self.name)) > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File > "/openstack/venvs/neutron-13.3.8/lib/python2.7/site- > packages/neutron/agent/linux/ip_lib.py", line 361, in _as_root > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent > use_root_namespace=use_root_namespace) > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File > "/openstack/venvs/neutron-13.3.8/lib/python2.7/site- > packages/neutron/agent/linux/ip_lib.py", line 94, in _as_root > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent > log_fail_as_error=self.log_fail_as_error) > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File > "/openstack/venvs/neutron-13.3.8/lib/python2.7/site- > packages/neutron/agent/linux/ip_lib.py", line 103, in _execute > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent > log_fail_as_error=log_fail_as_error) > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File > "/openstack/venvs/neutron-13.3.8/lib/python2.7/site- > packages/neutron/agent/linux/utils.py", line 140, in execute > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent raise > RuntimeError(msg) > 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent RuntimeError: > Exit code: 1; Stdin: ; Stdout: ; Stderr: Cannot find device "qg-c4080b5c-c9" > > Has anyone come across this before? > > We don't seem to have an entry for them anywhere from what we can see. > > Regards, > > > -- > Grant Morley > Cloud Lead > Absolute DevOps Ltd > Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP > www.absolutedevops.io gr...@absolutedevops.io <grant@absolutedevops.i> 0845 > 874 0580 > > > _______________________________________________ > OpenStack-operators mailing > listOpenStack-operators@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators > > > -- > Grant Morley > Cloud Lead > Absolute DevOps Ltd > Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP > www.absolutedevops.io gr...@absolutedevops.io <grant@absolutedevops.i> 0845 > 874 0580 > > _______________________________________________ > OpenStack-operators mailing list > OpenStack-operators@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators > >
_______________________________________________ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
