On Thu, May 3, 2012 at 10:31 AM, Gary Kotton <gkot...@redhat.com> wrote:
> **
> On 05/03/2012 08:13 PM, Dan Wendlandt wrote:
>
>
> I did the following:
> 1. rebooted the applaince
> 2. killed the mysql service
> 3. killed the quantum service
> 4. started the quantime service
> 5. when the service starts it exists with an exception at db.create_engine
> I should have been more specific. I provided the message from the quantum
> CLI and not the quantum log. My bad.
> Yes, this is an edge case. When we first encountered it we resolved it by
> starting the mysql service prior to the quantum. It would be best if the
> service tried to reconnect.
>
I see, we were talking about two different scenarios. Your point is that
the service fails to start if the DB is not running. Seems similar to the
issue you are fixing with the agents. Agree that this makes sense to fix.
Dan
>
> Thanks
> Gary
>
>
> Dan
>
>
> root@devstack-multi1:~# quantum list_nets default
> Virtual Networks for Tenant default
> Network ID: f6513015-2ba5-443b-8055-a2809cd34c20
> root@devstack-multi1:~# /etc/init.d/mysql stop
> root@devstack-multi1:~# mysql -uroot -pnova
> ERROR 2002 (HY000): Can't connect to local MySQL server through socket
> '/var/run/mysqld/mysqld.sock' (2)
> root@devstack-multi1:~# su stack
> stack@devstack-multi1:/root$ screen -x stack
>
> [1]+ Stopped screen -x stack
> stack@devstack-multi1:/root$ quantum list_nets default
> Traceback (most recent call last):
> File "/opt/stack/python-quantumclient/quantum/client/cli_lib.py", line
> 237, in list_nets_v11
> res = client.list_networks(**filters)
> File "/opt/stack/python-quantumclient/quantum/client/__init__.py", line
> 145, in with_params
> ret = self.function(instance, *args)
> File "/opt/stack/python-quantumclient/quantum/client/__init__.py", line
> 522, in list_networks
> return self.get(self.networks_path, params=filters)
> File "/opt/stack/python-quantumclient/quantum/client/__init__.py", line
> 379, in get
> headers=headers, params=params)
> File "/opt/stack/python-quantumclient/quantum/client/__init__.py", line
> 364, in retry_request
> headers=headers, params=params)
> File "/opt/stack/python-quantumclient/quantum/client/__init__.py", line
> 305, in do_request
> self._handle_fault_response(status_code, data)
> File "/opt/stack/python-quantumclient/quantum/client/__init__.py", line
> 228, in _handle_fault_response
> EXCEPTION_HANDLERS[self.version](status_code, des_error_body)
> File "/opt/stack/python-quantumclient/quantum/client/__init__.py", line
> 119, in exception_handler_v11
> raise exceptions.QuantumClientException(message=msg)
> QuantumClientException: 500-{'message': 'Traceback (most recent call
> last):\n File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line
> 336, in handle_one_response\n result = self.application(self.environ,
> start_response)\n File "/usr/lib/python2.7/dist-packages/paste/urlmap.py",
> line 203, in __call__\n return app(environ, start_response)\n File
> "/usr/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__\n
> return resp(environ, start_response)\n File
> "/usr/lib/pymodules/python2.7/routes/middleware.py", line 131, in
> __call__\n response = self.app(environ, start_response)\n File
> "/usr/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__\n
> return resp(environ, start_response)\n File
> "/usr/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__\n
> return resp(environ, start_response)\n File
> "/usr/lib/pymodules/python2.7/routes/middleware.py", line 131, in
> __call__\n response = self.app(environ, start_response)\n File
> "/usr/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__\n
> return resp(environ, start_response)\n File
> "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__\n
> resp = self.call_func(req, *args, **self.kwargs)\n File
> "/usr/lib/python2.7/dist-packages/webob/dec.py", line 208, in call_func\n
> return self.func(req, *args, **kwargs)\n File
> "/opt/stack/quantum/quantum/wsgi.py", line 748, in __call__\n
> action_result = self.dispatch(request, action, args)\n File
> "/opt/stack/quantum/quantum/wsgi.py", line 780, in dispatch\n return
> controller_method(request=request, **action_args)\n File
> "/opt/stack/quantum/quantum/api/api_common.py", line 108, in the_func\n
> return func(*args, **kwargs)\n File
> "/opt/stack/quantum/quantum/api/networks.py", line 101, in index\n
> return self._items(request, tenant_id)\n File
> "/opt/stack/quantum/quantum/api/networks.py", line 84, in _items\n
> filter_opts=filter_opts)\n File
> "/opt/stack/quantum/quantum/plugins/openvswitch/ovs_quantum_plugin.py",
> line 122, in get_all_networks\n for x in db.network_list(tenant_id):\n
> File "/opt/stack/quantum/quantum/db/api.py", line 131, in network_list\n
> filter_by(tenant_id=tenant_id).\\\n File
> "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.6-py2.7-linux-x86_64.egg/sqlalchemy/orm/query.py",
> line 2066, in all\n return list(self)\n File
> "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.6-py2.7-linux-x86_64.egg/sqlalchemy/orm/query.py",
> line 2176, in __iter__\n return self._execute_and_instances(context)\n
> File
> "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.6-py2.7-linux-x86_64.egg/sqlalchemy/orm/query.py",
> line 2189, in _execute_and_instances\n close_with_result=True)\n File
> "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.6-py2.7-linux-x86_64.egg/sqlalchemy/orm/query.py",
> line 2180, in _connection_from_session\n **kw)\n File
> "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.6-py2.7-linux-x86_64.egg/sqlalchemy/orm/session.py",
> line 729, in connection\n close_with_result=close_with_result)\n File
> "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.6-py2.7-linux-x86_64.egg/sqlalchemy/orm/session.py",
> line 735, in _connection_for_bind\n return
> engine.contextual_connect(**kwargs)\n File
> "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.6-py2.7-linux-x86_64.egg/sqlalchemy/engine/base.py",
> line 2476, in contextual_connect\n self.pool.connect(),\n File
> "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.6-py2.7-linux-x86_64.egg/sqlalchemy/pool.py",
> line 224, in connect\n return _ConnectionFairy(self).checkout()\n File
> "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.6-py2.7-linux-x86_64.egg/sqlalchemy/pool.py",
> line 466, in checkout\n self.connection =
> self._connection_record.get_connection()\n File
> "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.6-py2.7-linux-x86_64.egg/sqlalchemy/pool.py",
> line 300, in get_connection\n self.connection = self.__connect()\n File
> "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.6-py2.7-linux-x86_64.egg/sqlalchemy/pool.py",
> line 330, in __connect\n connection = self.__pool._creator()\n File
> "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.6-py2.7-linux-x86_64.egg/sqlalchemy/engine/strategies.py",
> line 80, in connect\n return dialect.connect(*cargs, **cparams)\n File
> "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.6-py2.7-linux-x86_64.egg/sqlalchemy/engine/default.py",
> line 285, in connect\n return self.dbapi.connect(*cargs, **cparams)\n
> File "/usr/lib/pymodules/python2.7/MySQLdb/__init__.py", line 81, in
> Connect\n return Connection(*args, **kwargs)\n File
> "/usr/lib/pymodules/python2.7/MySQLdb/connections.py", line 187, in
> __init__\n super(Connection, self).__init__(*args,
> **kwargs2)\nOperationalError: (OperationalError) (2002, "Can\'t connect to
> local MySQL server through socket \'/var/run/mysqld/mysqld.sock\' (2)")
> None None\n'}
> stack@devstack-multi1:/root$ screen -x stack
>
> [2]+ Stopped screen -x stack
> stack@devstack-multi1:/root$ sudo service mysql start
> mysql start/running, process 27677
> stack@devstack-multi1:/root$ quantum list_nets default
> Virtual Networks for Tenant default
> Network ID: f6513015-2ba5-443b-8055-a2809cd34c20
>
>
>
>
>
>
>
>
>
>
>>
>> One improvement we definitely need is to hide such errors from the tenant
>> (who should just get generic errors). We had a bug filed on that already,
>> and Guilherme proposed a patch for it yesterday:
>> https://review.openstack.org/#/c/7019/ . Perhaps that is what you're
>> suggesting below?
>>
>> Brilliant - this addresses part of it (saved a few -1's for me :))
>>
>>
>> Thanks!
>>
>> dan
>>
>> >
>> > 3. Error handling - for example when using the CLI and the DB is down
>> we get the attached output. It would be better if the trace was looged and
>> the user would receive the specific error message.
>> > [f17-dev@localhost ~]$ quantum list_nets default
>> > Traceback (most recent call last):
>> > File "/usr/lib/python2.7/site-packages/quantum/client/cli_lib.py",
>> line 237, in list_nets_v11
>> > res = client.list_networks(**filters)
>> > File
>> "/usr/lib/python2.7/site-packages/quantum/client/__init__.py", line 145, in
>> with_params
>> > ret = self.function(instance, *args)
>> > File
>> "/usr/lib/python2.7/site-packages/quantum/client/__init__.py", line 522, in
>> list_networks
>> > ...
>> > raise exceptions.QuantumClientException(message=msg)
>> > QuantumClientException: 500-{'message': 'Traceback (most recent call
>> last):\n File "/usr/lib/python2.7/site-packages/eventlet/wsgi.py", line
>> 336, in handle_one_response\n result =
>> self.application(self.environ, start_response)\n File
>> "/usr/lib/python2.7/site-packages/paste/urlmap.py", line 203, in __call__\n
>> return app(environ, start_response)\n File
>> "/usr/lib/python2.7/site-packages/webob/dec.py", line 159, in __call__\n
>> return resp(environ, start_response)\n File
>> "/usr/lib/python2.7/site-packages/routes/middleware.py", line 131, in
>> __call__\n response = self.app(environ, start_response)\n File
>> "/usr/lib/python2.7/site-packages/webob/dec.py", line 159, in __call__\n
>> return resp(environ, start_response)\n File
>> "/usr/lib/python2.7/site-packages/webob/dec.py", line 159, in __call__\n
>> return resp(environ, start_response)\n File
>> "/usr/lib/python2.7/site-packages/routes/middleware.py", line 131, in
>> __call__\n response = self.app(environ, start_response)\n File
>> "/usr/lib/python2.7/site-packages/webob/dec.py", line 159, in __call__\n
>> return resp(environ, start_response)\n File
>> "/usr/lib/python2.7/site-packages/webob/dec.py", line 147, in __call__\n
>> resp = self.call_func(req, *args, **self.kwargs)\n File
>> "/usr/lib/python2.7/site-packages/webob/dec.py", line 210, in call_func\n
>> return self.func(req, *args, **kwargs)\n File
>> "/usr/lib/python2.7/site-packages/quantum/wsgi.py", line 748, in __call__\n
>> action_result = self.dispatch(request, action, args)\n File
>> "/usr/lib/python2.7/site-packages/quantum/wsgi.py", line 780, in dispatch\n
>> return controller_method(request=request, **action_args)\n File
>> "/usr/lib/python2.7/site-packages/quantum/api/api_common.py", line 108, in
>> the_func\n return func(*args, **kwargs)\n File
>> "/usr/lib/python2.7/site-packages/quantum/api/networks.py", line 101, in
>> index\n return self._items(request, tenant_id)\n File
>> "/usr/lib/python2.7/site-packages/quantum/api/networks.py", line 84, in
>> _items\n filter_opts=filter_opts)\n File
>> "/usr/lib/python2.7/site-packages/quantum/plugins/linuxbridge/LinuxBridgePlugin.py",
>> line 71, in get_all_networks\n networks_list =
>> db.network_list(tenant_id)\n File
>> "/usr/lib/python2.7/site-packages/quantum/db/api.py", line 131, in
>> network_list\n filter_by(tenant_id=tenant_id).\\\n File
>> "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2066, in
>> all\n return list(self)\n File
>> "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2176, in
>> __iter__\n return self._execute_and_instances(context)\n File
>> "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2189, in
>> _execute_and_instances\n close_with_result=True)\n File
>> "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2180, in
>> _connection_from_session\n **kw)\n File
>> "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 729,
>> in connection\n close_with_result=close_with_result)\n File
>> "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 735,
>> in _connection_for_bind\n return engine.contextual_connect(**kwargs)\n
>> File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line
>> 2476, in contextual_connect\n self.pool.connect(),\n File
>> "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 224, in
>> connect\n return _ConnectionFairy(self).checkout()\n File
>> "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 466, in
>> checkout\n self.connection = self._connection_record.get_connection()\n
>> File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 300, in
>> get_connection\n self.connection = self.__connect()\n File
>> "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 330, in
>> __connect\n connection = self.__pool._creator()\n File
>> "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line
>> 80, in connect\n return dialect.connect(*cargs, **cparams)\n File
>> "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line
>> 285, in connect\n return self.dbapi.connect(*cargs, **cparams)\n File
>> "/usr/lib64/python2.7/site-packages/MySQLdb/__init__.py", line 81, in
>> Connect\n return Connection(*args, **kwargs)\n File
>> "/usr/lib64/python2.7/site-packages/MySQLdb/connections.py", line 187, in
>> __init__\n super(Connection, self).__init__(*args,
>> **kwargs2)\nOperationalError: (OperationalError) (2003, "Can\'t connect to
>> MySQL server on \'localhost.localdomain\' (111)") None None\n'}
>> >
>> > Any thought and or comments. Any objections if I update the blueprint
>> to deal with the the above items.
>> > Thanks
>> > Gary
>> >
>> >
>> > --
>> > Mailing list: https://launchpad.net/~netstack
>> > Post to : netstack@lists.launchpad.net
>> > Unsubscribe : https://launchpad.net/~netstack
>> > More help : https://help.launchpad.net/ListHelp
>>
>>
>>
>>
>> --
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> Dan Wendlandt
>> Nicira, Inc: www.nicira.com
>> twitter: danwendlandt
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>
>>
>>
>
>
> --
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Dan Wendlandt
> Nicira, Inc: www.nicira.com
> twitter: danwendlandt
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>
>
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Dan Wendlandt
Nicira, Inc: www.nicira.com
twitter: danwendlandt
~~~~~~~~~~~~~~~~~~~~~~~~~~~
--
Mailing list: https://launchpad.net/~netstack
Post to : netstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~netstack
More help : https://help.launchpad.net/ListHelp
