Public bug reported: May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager [None req-b332b601-98dc-4f12-9235-6cf5943fb71e cinder_grenade cinder_grenade] Error during notification for neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver._delete_security_group-3711357 security_group, after_delete: ovsdbapp.exceptions.TimeoutException: Commands [PgDelCommand(_result=None, name=pg_cf5c9692_6de1_4186_b64f_5317563d14da, if_exists=True)] exceeded timeout 180 seconds, cause: Result queue is empty 55001 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager Traceback (most recent call last): 55002 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 54, in commit 55003 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager result = self.results.get(timeout=self.timeout) 55004 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/queue.py", line 320, in get 55005 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager try: 55006 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/queue.py", line 139, in wait 55007 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager try: 55008 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch 55009 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager return self.greenlet.switch() 55010 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager _queue.Empty 55011 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager 55012 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager The above exception was the direct cause of the following exception: 55013 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager 55014 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager Traceback (most recent call last): 55015 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/callbacks/manager.py", line 189, in _notify_loop 55016 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager callback.method(resource, event, trigger, payload=payload) 55017 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/opt/stack/old/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 430, in _delete_security_group 55018 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager self._ovn_client.delete_security_group(context, 55019 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/opt/stack/old/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2583, in delete_security_group 55020 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager with self._nb_idl.transaction(check_error=True) as txn: 55021 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ 55022 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager next(self.gen) 55023 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/opt/stack/old/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 274, in transaction 55024 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager with super(OvsdbNbOvnIdl, self).transaction(*args, **kwargs) as t: 55025 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ 55026 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager next(self.gen) 55027 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/api.py", line 114, in transaction 55028 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager with self.create_transaction( 55029 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/api.py", line 71, in __exit__ 55030 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager self.result = self.commit() 55031 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 56, in commit 55032 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager raise exceptions.TimeoutException( 55033 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager ovsdbapp.exceptions.TimeoutException: Commands [PgDelCommand(_result=None, name=pg_cf5c9692_6de1_4186_b64f_5317563d14da, if_exists=True)] exceeded timeout 180 seconds, cause: Result queue is empty 55034 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager
https://zuul.opendev.org/t/openstack/build/34e1c6cf11e04d17bbac42bf399d76b5/log/controller/logs/screen- neutron-api.txt seen during grenade: 025-05-19 10:35:51.109 | + /opt/stack/new/grenade/projects/70_cinder/resources.sh:destroy:289 : openstack security group delete cinder_grenade 14212 2025-05-19 10:40:52.922 | Failed to delete group with name or ID 'cinder_grenade': HttpException: 500: Server Error for url: http://200.225.47.46/networking/v2.0/security-groups/cf5c9692-6de1-4186-b64f-5317563d14da, 500 Internal Server Error: Internal Server Error: The server encountered an internal error or: misconfiguration and was unable to complete: your request.: Please contact the server administrator at: webmaster@localhost to inform them of the time this error occurred,: and the actions you performed just before this error.: More information about this error may be available: in the server error log.: Apache/2.4.52 (Ubuntu) Server at 200.225.47.46 Port 80 14213 2025-05-19 10:40:52.922 | 1 of 1 groups failed to delete. https://zuul.opendev.org/t/openstack/build/34e1c6cf11e04d17bbac42bf399d76b5/log/controller/logs/grenade.sh_log.txt#14211-14213 ** Affects: neutron Importance: Undecided Status: New ** Tags: gate-failure ** Tags added: gate-failure -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2111272 Title: security group delete fails with PgDelCommand.*exceeded timeout 180 seconds, cause: Result queue is empty Status in neutron: New Bug description: May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager [None req-b332b601-98dc-4f12-9235-6cf5943fb71e cinder_grenade cinder_grenade] Error during notification for neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver._delete_security_group-3711357 security_group, after_delete: ovsdbapp.exceptions.TimeoutException: Commands [PgDelCommand(_result=None, name=pg_cf5c9692_6de1_4186_b64f_5317563d14da, if_exists=True)] exceeded timeout 180 seconds, cause: Result queue is empty 55001 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager Traceback (most recent call last): 55002 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 54, in commit 55003 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager result = self.results.get(timeout=self.timeout) 55004 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/queue.py", line 320, in get 55005 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager try: 55006 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/queue.py", line 139, in wait 55007 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager try: 55008 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 310, in switch 55009 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager return self.greenlet.switch() 55010 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager _queue.Empty 55011 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager 55012 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager The above exception was the direct cause of the following exception: 55013 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager 55014 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager Traceback (most recent call last): 55015 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/callbacks/manager.py", line 189, in _notify_loop 55016 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager callback.method(resource, event, trigger, payload=payload) 55017 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/opt/stack/old/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 430, in _delete_security_group 55018 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager self._ovn_client.delete_security_group(context, 55019 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/opt/stack/old/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2583, in delete_security_group 55020 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager with self._nb_idl.transaction(check_error=True) as txn: 55021 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ 55022 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager next(self.gen) 55023 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/opt/stack/old/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 274, in transaction 55024 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager with super(OvsdbNbOvnIdl, self).transaction(*args, **kwargs) as t: 55025 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ 55026 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager next(self.gen) 55027 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/api.py", line 114, in transaction 55028 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager with self.create_transaction( 55029 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/api.py", line 71, in __exit__ 55030 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager self.result = self.commit() 55031 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager File "/opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 56, in commit 55032 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager raise exceptions.TimeoutException( 55033 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager ovsdbapp.exceptions.TimeoutException: Commands [PgDelCommand(_result=None, name=pg_cf5c9692_6de1_4186_b64f_5317563d14da, if_exists=True)] exceeded timeout 180 seconds, cause: Result queue is empty 55034 May 19 10:38:53.079114 np0040803311 devstack@neutron-api.service[75684]: ERROR neutron_lib.callbacks.manager https://zuul.opendev.org/t/openstack/build/34e1c6cf11e04d17bbac42bf399d76b5/log/controller/logs/screen- neutron-api.txt seen during grenade: 025-05-19 10:35:51.109 | + /opt/stack/new/grenade/projects/70_cinder/resources.sh:destroy:289 : openstack security group delete cinder_grenade 14212 2025-05-19 10:40:52.922 | Failed to delete group with name or ID 'cinder_grenade': HttpException: 500: Server Error for url: http://200.225.47.46/networking/v2.0/security-groups/cf5c9692-6de1-4186-b64f-5317563d14da, 500 Internal Server Error: Internal Server Error: The server encountered an internal error or: misconfiguration and was unable to complete: your request.: Please contact the server administrator at: webmaster@localhost to inform them of the time this error occurred,: and the actions you performed just before this error.: More information about this error may be available: in the server error log.: Apache/2.4.52 (Ubuntu) Server at 200.225.47.46 Port 80 14213 2025-05-19 10:40:52.922 | 1 of 1 groups failed to delete. https://zuul.opendev.org/t/openstack/build/34e1c6cf11e04d17bbac42bf399d76b5/log/controller/logs/grenade.sh_log.txt#14211-14213 To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2111272/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
