Public bug reported: OpenStack stable/kilo release, Ubuntu 14.04 devstack setup .while multiple calls are generated during tempest run(tempest.api.networks) in HP networking CI setup we see queuePool limit related exception which results in tempest failure.
tempest tests that are run include cd /opt/stack/tempest; testr init ; ./tools/pretty_tox_serial.sh \" tempest.api.network.test_networks tempest.api.network.test_extensions tempest.api.network.test_allowed_address_pair tempest.api.network.test_extra_dhcp_options tempest.api.network.test_floating_ips_negative tempest.api.network.test_floating_ips tempest.api.network.test_networks_negative tempest.api.network.test_networks tempest.api.network.test_ports tempest.api.network.test_routers_negative tempest.api.network.test_routers tempest.api.network.test_security_groups_negative tempest.api.network.test_security_groups tempest.api.network.test_service_type_management tempest.scenario.test_network_basic_ops|^test_security_groups_basic_ops|^test_server_basic_ops|^test_network_advanced_server_ops|^test_server_advanced_ops\" " detailed logs can be found at http://15.126.220.115/OVSVAPP-VLAN-KILO/602/203586/9215e7309301fd72961e906a1ae5afe1d5c8f909/ might have been deleted concurrently. 2015-07-21 12:27:55.595 WARNING neutron.db.agentschedulers_db [req-2585e7c4-8f33-439b-8b64-afc1522a686a None None] Removing network 17d6770a-c478-46d4-9556-ee77bbaf2d6b from agent 04fa72de-c1f9-4b51-8369-3b027aaf42d4 because the agent did not report to the server in the last 150 seconds. 2015-07-21 12:28:15.822 ERROR oslo_messaging.rpc.dispatcher [req-99ec4c7c-3f9d-4dd7-b920-c57606a07df7 None None] Exception during message handling: QueuePool limit of size 10 overflow 20 reached, connection timed out, timeout 10 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last): 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher executor_callback)) 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher executor_callback) 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args) 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/neutron/neutron/db/agents_db.py", line 281, in report_state 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher self.plugin.create_or_update_agent(context, agent_state) 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/neutron/neutron/db/agents_db.py", line 238, in create_or_update_agent 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher return self._create_or_update_agent(context, agent) 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/neutron/neutron/db/agents_db.py", line 217, in _create_or_update_agent 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher context, agent['agent_type'], agent['host']) 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/neutron/neutron/db/agents_db.py", line 193, in _get_agent_by_type_and_host 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher Agent.host == host).one() 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2395, in one 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher ret = list(self) 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2438, in __iter__ 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher return self._execute_and_instances(context) 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2451, in _execute_and_instances 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher close_with_result=True) 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2442, in _connection_from_session 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher **kw) 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 854, in connection 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher close_with_result=close_with_result) 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 858, in _connection_for_bind 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher return self.transaction._connection_for_bind(engine) 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 322, in _connection_for_bind 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher conn = bind.contextual_connect() 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_db/sqlalchemy/compat/handle_error.py", line 208, in contextual_connect 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher self._wrap_pool_connect(self.pool.connect, None), 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_db/sqlalchemy/compat/handle_error.py", line 215, in _wrap_pool_connect 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher return fn() 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 338, in connect 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher return _ConnectionFairy._checkout(self) 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 644, in _checkout 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher fairy = _ConnectionRecord.checkout(pool) 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 440, in checkout 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher rec = pool._do_get() 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 959, in _do_get 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher (self.size(), self.overflow(), self._timeout)) 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher TimeoutError: QueuePool limit of size 10 overflow 20 reached, connection timed out, timeout 10 2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher 2015-07-21 12:28:15.885 ERROR oslo_messaging.rpc.dispatcher [req-6ee592e3-e5aa-4d6d-a422-3e7ca4734a25 None None] Exception during message handling: QueuePool limit of size 10 overflow 20 reached, connection timed out, timeout 10 ** Affects: neutron (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1477077 Title: Exception during message handling: QueuePool limit of size 10 overflow 20 reached, connection timed out, timeout 10 stable/kilo To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1477077/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
