Reviewed: https://review.opendev.org/c/openstack/neutron/+/839779
Committed:
https://opendev.org/openstack/neutron/commit/8df2f69b6f93fd964cc96efb42d085063b90a8e6
Submitter: "Zuul (22348)"
Branch: master
commit 8df2f69b6f93fd964cc96efb42d085063b90a8e6
Author: Slawek Kaplonski <skapl...@redhat.com>
Date: Thu Apr 28 16:51:25 2022 +0200
Handle properly ObjectNotFound while deleting network from DHCP agent
In case when 2 neutron servers are trying to remove network from the
same DHCP agent it may happend that one of them will not be able
to find binding object anymore thus deletion of that binding object
will raise ObjectNotFound exception.
This patch adds proper handling of such case to not raise ugly
stacktrace in neutron logs.
Closes-Bug: #1970759
Change-Id: I67d516c4583aa0c20416114b92a6d69ece5b970c
** Changed in: neutron
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1970759
Title:
Race during removal of the network from DHCP agent
Status in neutron:
Fix Released
Bug description:
It may happen that 2 neutron-server workers will try to remove network
from the DHCP agent in almost the same way. On one of them all will
work fine:
2022-04-19 18:51:26.579 50 WARNING neutron.db.agentschedulers_db
[req-73439ae5-d6a4-432c-92f0-872cf5d19bec - - - - -] Removing network
d2ee3359-3ddb-44ba-ba89-68c8698d550a from agent
1ddb078b-8cfa-48b4-8a87-228f32f7a947 because the agent did not report
to the server in the last 150 seconds.
but on second one it will fail with error:
2022-04-19 18:51:26.634 50 WARNING neutron.db.agentschedulers_db
[req-ee6aab2f-14e8-4080-a1a6-841421cb28e9 - - - - -] Removing network
d2ee3359-3ddb-44ba-ba89-68c8698d550a from agent
1ddb078b-8cfa-48b4-8a87-228f32f7a947 because the agent did not report to the
server in the last 150 seconds.
2022-04-19 18:51:26.732 50 ERROR neutron.db.agentschedulers_db
[req-ee6aab2f-14e8-4080-a1a6-841421cb28e9 - - - - -] Unexpected exception
occurred while removing network d2ee3359-3ddb-44ba-ba89-68c8698d550a from agent
1ddb078b-8cfa-48b4-8a87-228f32f7a947: neutron_lib.exceptions.ObjectNotFound:
Object NetworkDhcpAgentBinding(network_id=d2ee3359-3ddb-44ba-ba89-68c8698d550a,
dhcp_agent_id=1ddb078b-8cfa-48b4-8a87-228f32f7a947) not found.
2022-04-19 18:51:26.732 50 ERROR neutron.db.agentschedulers_db Traceback
(most recent call last):
2022-04-19 18:51:26.732 50 ERROR neutron.db.agentschedulers_db File
"/usr/lib/python3.6/site-packages/neutron/db/agentschedulers_db.py", line 342,
in remove_networks_from_down_agents
2022-04-19 18:51:26.732 50 ERROR neutron.db.agentschedulers_db
notify=False)
2022-04-19 18:51:26.732 50 ERROR neutron.db.agentschedulers_db File
"/usr/lib/python3.6/site-packages/neutron/db/agentschedulers_db.py", line 431,
in remove_network_from_dhcp_agent
2022-04-19 18:51:26.732 50 ERROR neutron.db.agentschedulers_db
binding_obj.delete()
2022-04-19 18:51:26.732 50 ERROR neutron.db.agentschedulers_db File
"/usr/lib/python3.6/site-packages/neutron/objects/base.py", line 912, in delete
2022-04-19 18:51:26.732 50 ERROR neutron.db.agentschedulers_db
self._get_composite_keys()))
2022-04-19 18:51:26.732 50 ERROR neutron.db.agentschedulers_db File
"/usr/lib/python3.6/site-packages/neutron/objects/db/api.py", line 94, in
delete_object
2022-04-19 18:51:26.732 50 ERROR neutron.db.agentschedulers_db db_obj =
_safe_get_object(obj_cls, context, **kwargs)
2022-04-19 18:51:26.732 50 ERROR neutron.db.agentschedulers_db File
"/usr/lib/python3.6/site-packages/neutron/objects/db/api.py", line 80, in
_safe_get_object
2022-04-19 18:51:26.732 50 ERROR neutron.db.agentschedulers_db
id="%s(%s)" % (obj_cls.db_model.__name__, key))
2022-04-19 18:51:26.732 50 ERROR neutron.db.agentschedulers_db
neutron_lib.exceptions.ObjectNotFound: Object
NetworkDhcpAgentBinding(network_id=d2ee3359-3ddb-44ba-ba89-68c8698d550a,
dhcp_agent_id=1ddb078b-8cfa-48b4-8a87-228f32f7a947) not found.
2022-04-19 18:51:26.732 50 ERROR neutron.db.agentschedulers_db
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1970759/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
