Public bug reported:
Version : Havana
I have two controllers in my environment, and deploy glance-api on each
controller.
In nova.conf :
glance_api_servers=controller2:9292,controller2:9292
glance_num_retries = 2
When I kill the glance on controller2, then run "nova image-list", I will get
error sometimes:
# nova image-list
+--------------------------------------+--------+--------+--------+
| ID | Name | Status | Server |
+--------------------------------------+--------+--------+--------+
| 668a4e66-97d9-40d7-888e-dd9db53438c4 | centos | ACTIVE | |
| 7e7917a6-96c5-4000-b167-efc13be4124c | cirros | ACTIVE | |
+--------------------------------------+--------+--------+--------+
# nova image-list
ERROR: The server has either erred or is incapable of performing the requested
operation. (HTTP 500) (Request-ID: req-09d764df-4e8b-4330-b39e-14e4736c2e68)
The error in /var/log/nova/api.log is:
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack Traceback (most recent
call last):
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/nova/api/openstack/__init__.py", line 119, in
__call__
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack return
req.get_response(self.application)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/webob/request.py", line 1296, in send
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack application,
catch_exc_info=False)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/webob/request.py", line 1260, in
call_application
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack app_iter =
application(self.environ, start_response)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/webob/dec.py", line 144, in __call__
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack return resp(environ,
start_response)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py",
line 574, in __call__
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack return self.app(env,
start_response)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/webob/dec.py", line 144, in __call__
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack return resp(environ,
start_response)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/webob/dec.py", line 144, in __call__
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack return resp(environ,
start_response)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/Routes-1.12.3-py2.6.egg/routes/middleware.py",
line 131, in __call__
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack response =
self.app(environ, start_response)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/webob/dec.py", line 144, in __call__
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack return resp(environ,
start_response)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/webob/dec.py", line 130, in __call__
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack resp =
self.call_func(req, *args, **self.kwargs)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/webob/dec.py", line 195, in call_func
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack return self.func(req,
*args, **kwargs)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/nova/api/openstack/wsgi.py", line 917, in
__call__
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack content_type, body,
accept)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/nova/api/openstack/wsgi.py", line 976, in
_process_stack
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack action_result =
self.dispatch(meth, request, action_args)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/nova/api/openstack/wsgi.py", line 1057, in
dispatch
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack return
method(req=request, **action_args)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/nova/api/openstack/compute/images.py", line
203, in detail
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack **page_params)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/nova/image/glance.py", line 264, in detail
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack for image in images:
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/glanceclient/v1/images.py", line 174, in
paginate
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack images =
self._list(url, "images")
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/glanceclient/common/base.py", line 53, in
_list
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack resp, body =
self.api.json_request('GET', url)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/glanceclient/common/http.py", line 266, in
json_request
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack resp, body_iter =
self._http_request(url, method, **kwargs)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/glanceclient/common/http.py", line 235, in
_http_request
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack raise
exc.CommunicationError(message=message)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack CommunicationError: Error
communicating with http://10.224.159.75:9292 [Errno 111] ECONNREFUSED
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack
2014-01-22 05:33:08.572 8866 INFO nova.api.openstack
[req-5d5b5dd1-36a5-48d9-8d6e-21be5551b135 feea78294ac84fb29583327109e899d6
3a5541d10e6b4a0fbcd45213e0e17ada]
http://10.224.159.111/v2/3a5541d10e6b4a0fbcd45213e0e17ada/images/detail
returned with HTTP 500
"10.224.159.75:9292" is my controller2 IP.
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1271417
Title:
glance_num_retries can't work
Status in OpenStack Compute (Nova):
New
Bug description:
Version : Havana
I have two controllers in my environment, and deploy glance-api on each
controller.
In nova.conf :
glance_api_servers=controller2:9292,controller2:9292
glance_num_retries = 2
When I kill the glance on controller2, then run "nova image-list", I will get
error sometimes:
# nova image-list
+--------------------------------------+--------+--------+--------+
| ID | Name | Status | Server |
+--------------------------------------+--------+--------+--------+
| 668a4e66-97d9-40d7-888e-dd9db53438c4 | centos | ACTIVE | |
| 7e7917a6-96c5-4000-b167-efc13be4124c | cirros | ACTIVE | |
+--------------------------------------+--------+--------+--------+
# nova image-list
ERROR: The server has either erred or is incapable of performing the
requested operation. (HTTP 500) (Request-ID:
req-09d764df-4e8b-4330-b39e-14e4736c2e68)
The error in /var/log/nova/api.log is:
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack Traceback (most recent
call last):
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/nova/api/openstack/__init__.py", line 119, in
__call__
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack return
req.get_response(self.application)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/webob/request.py", line 1296, in send
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack application,
catch_exc_info=False)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/webob/request.py", line 1260, in
call_application
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack app_iter =
application(self.environ, start_response)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/webob/dec.py", line 144, in __call__
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack return
resp(environ, start_response)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py",
line 574, in __call__
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack return
self.app(env, start_response)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/webob/dec.py", line 144, in __call__
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack return
resp(environ, start_response)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/webob/dec.py", line 144, in __call__
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack return
resp(environ, start_response)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/Routes-1.12.3-py2.6.egg/routes/middleware.py",
line 131, in __call__
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack response =
self.app(environ, start_response)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/webob/dec.py", line 144, in __call__
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack return
resp(environ, start_response)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/webob/dec.py", line 130, in __call__
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack resp =
self.call_func(req, *args, **self.kwargs)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/webob/dec.py", line 195, in call_func
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack return
self.func(req, *args, **kwargs)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/nova/api/openstack/wsgi.py", line 917, in
__call__
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack content_type, body,
accept)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/nova/api/openstack/wsgi.py", line 976, in
_process_stack
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack action_result =
self.dispatch(meth, request, action_args)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/nova/api/openstack/wsgi.py", line 1057, in
dispatch
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack return
method(req=request, **action_args)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/nova/api/openstack/compute/images.py", line
203, in detail
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack **page_params)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/nova/image/glance.py", line 264, in detail
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack for image in images:
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/glanceclient/v1/images.py", line 174, in
paginate
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack images =
self._list(url, "images")
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/glanceclient/common/base.py", line 53, in
_list
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack resp, body =
self.api.json_request('GET', url)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/glanceclient/common/http.py", line 266, in
json_request
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack resp, body_iter =
self._http_request(url, method, **kwargs)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack File
"/usr/lib/python2.6/site-packages/glanceclient/common/http.py", line 235, in
_http_request
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack raise
exc.CommunicationError(message=message)
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack CommunicationError:
Error communicating with http://10.224.159.75:9292 [Errno 111] ECONNREFUSED
2014-01-22 05:33:08.568 8866 TRACE nova.api.openstack
2014-01-22 05:33:08.572 8866 INFO nova.api.openstack
[req-5d5b5dd1-36a5-48d9-8d6e-21be5551b135 feea78294ac84fb29583327109e899d6
3a5541d10e6b4a0fbcd45213e0e17ada]
http://10.224.159.111/v2/3a5541d10e6b4a0fbcd45213e0e17ada/images/detail
returned with HTTP 500
"10.224.159.75:9292" is my controller2 IP.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1271417/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
