Public bug reported: When running rally scenario NeutronNetworks.create_and_delete_networks with concurrency of 60 the following error is observed:
--8<--8<--8<-- 2021-08-16 11:28:41.526 710 ERROR oslo_db.api [req-61e1d9da-1bad-4410-94ce-d2945c13a2d5 05971ba84eac4b8eb176bd935909f9d0 03904310315c47c7b33178da2bfc99a2 - default default] DB exceeded retry limit.: oslo_db.exception.RetryRequest: Unable to create the network. No available network found in maximum allowed attempts. 2021-08-16 11:28:41.526 710 ERROR oslo_db.api Traceback (most recent call last): 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper 2021-08-16 11:28:41.526 710 ERROR oslo_db.api return f(*args, **kwargs) 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped 2021-08-16 11:28:41.526 710 ERROR oslo_db.api LOG.debug("Retry wrapper got retriable exception: %s", e) 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2021-08-16 11:28:41.526 710 ERROR oslo_db.api self.force_reraise() 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2021-08-16 11:28:41.526 710 ERROR oslo_db.api six.reraise(self.type_, self.value, self.tb) 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/six.py", line 703, in reraise 2021-08-16 11:28:41.526 710 ERROR oslo_db.api raise value 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped 2021-08-16 11:28:41.526 710 ERROR oslo_db.api return f(*dup_args, **dup_kwargs) 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 1053, in create_network 2021-08-16 11:28:41.526 710 ERROR oslo_db.api result, mech_context = self._create_network_db(context, network) 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 1012, in _create_network_db 2021-08-16 11:28:41.526 710 ERROR oslo_db.api tenant_id) 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/managers.py", line 226, in create_network_segments 2021-08-16 11:28:41.526 710 ERROR oslo_db.api context, filters=filters) 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/managers.py", line 312, in _allocate_tenant_net_segment 2021-08-16 11:28:41.526 710 ERROR oslo_db.api segment = self._allocate_segment(context, network_type, filters) 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/managers.py", line 308, in _allocate_segment 2021-08-16 11:28:41.526 710 ERROR oslo_db.api return driver.obj.allocate_tenant_segment(context, filters) 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/type_tunnel.py", line 391, in allocate_tenant_segment 2021-08-16 11:28:41.526 710 ERROR oslo_db.api alloc = self.allocate_partially_specified_segment(context, **filters) 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/helpers.py", line 153, in allocate_partially_specified_segment 2021-08-16 11:28:41.526 710 ERROR oslo_db.api exceptions.NoNetworkFoundInMaximumAllowedAttempts()) 2021-08-16 11:28:41.526 710 ERROR oslo_db.api oslo_db.exception.RetryRequest: Unable to create the network. No available network found in maximum allowed attempts. 2021-08-16 11:28:41.526 710 ERROR oslo_db.api --8<--8<--8<-- ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1940073 Title: "Unable to create the network. No available network found in maximum allowed attempts." during rally stress test Status in neutron: New Bug description: When running rally scenario NeutronNetworks.create_and_delete_networks with concurrency of 60 the following error is observed: --8<--8<--8<-- 2021-08-16 11:28:41.526 710 ERROR oslo_db.api [req-61e1d9da-1bad-4410-94ce-d2945c13a2d5 05971ba84eac4b8eb176bd935909f9d0 03904310315c47c7b33178da2bfc99a2 - default default] DB exceeded retry limit.: oslo_db.exception.RetryRequest: Unable to create the network. No available network found in maximum allowed attempts. 2021-08-16 11:28:41.526 710 ERROR oslo_db.api Traceback (most recent call last): 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper 2021-08-16 11:28:41.526 710 ERROR oslo_db.api return f(*args, **kwargs) 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped 2021-08-16 11:28:41.526 710 ERROR oslo_db.api LOG.debug("Retry wrapper got retriable exception: %s", e) 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2021-08-16 11:28:41.526 710 ERROR oslo_db.api self.force_reraise() 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2021-08-16 11:28:41.526 710 ERROR oslo_db.api six.reraise(self.type_, self.value, self.tb) 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/six.py", line 703, in reraise 2021-08-16 11:28:41.526 710 ERROR oslo_db.api raise value 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped 2021-08-16 11:28:41.526 710 ERROR oslo_db.api return f(*dup_args, **dup_kwargs) 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 1053, in create_network 2021-08-16 11:28:41.526 710 ERROR oslo_db.api result, mech_context = self._create_network_db(context, network) 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 1012, in _create_network_db 2021-08-16 11:28:41.526 710 ERROR oslo_db.api tenant_id) 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/managers.py", line 226, in create_network_segments 2021-08-16 11:28:41.526 710 ERROR oslo_db.api context, filters=filters) 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/managers.py", line 312, in _allocate_tenant_net_segment 2021-08-16 11:28:41.526 710 ERROR oslo_db.api segment = self._allocate_segment(context, network_type, filters) 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/managers.py", line 308, in _allocate_segment 2021-08-16 11:28:41.526 710 ERROR oslo_db.api return driver.obj.allocate_tenant_segment(context, filters) 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/type_tunnel.py", line 391, in allocate_tenant_segment 2021-08-16 11:28:41.526 710 ERROR oslo_db.api alloc = self.allocate_partially_specified_segment(context, **filters) 2021-08-16 11:28:41.526 710 ERROR oslo_db.api File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/helpers.py", line 153, in allocate_partially_specified_segment 2021-08-16 11:28:41.526 710 ERROR oslo_db.api exceptions.NoNetworkFoundInMaximumAllowedAttempts()) 2021-08-16 11:28:41.526 710 ERROR oslo_db.api oslo_db.exception.RetryRequest: Unable to create the network. No available network found in maximum allowed attempts. 2021-08-16 11:28:41.526 710 ERROR oslo_db.api --8<--8<--8<-- To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1940073/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp