Fix proposed to branch: master
Review: https://review.openstack.org/430208
** Changed in: openstack-ansible
Status: Won't Fix => In Progress
** Changed in: openstack-ansible
Assignee: (unassigned) => Jean-Philippe Evrard (jean-philippe-evrard)
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Dashboard (Horizon).
https://bugs.launchpad.net/bugs/1624791
Title:
Horizon randomly fails to connect to the service APIs
Status in OpenStack Dashboard (Horizon):
New
Status in openstack-ansible:
In Progress
Bug description:
This started occurring after upgrading to RC1. Before that, I was
using Horizon's e7b4bdfe5d576766b34bf00cea3dcbcb42436420 plus cherry-
picked changesets I2db4218e7351e0017a7a74114be6ac7af803476c and
Idb58cebefab747f204e54ea6350db0852aec60f5.
Running nova, cinder, etc. from the CLI seems to work perfectly.
However Horizon randomly fails with "bad handshake: SysCallError(0,
None)" when connecting to other services, including Keystone.
The symptoms are:
* Login is intermittent, it randomly succeeds or fails.
* After being lucky enough to login, dashboards would randomly fail to
display information.
I tried to isolate and see if some of my 3 infra nodes was the
culprit, without success.
I also tried to destroy and reconstruct the horizon and keystone
containers multiple times without success.
I didn't try yet to downgrade Horizon to the
e7b4bdfe5d576766b34bf00cea3dcbcb42436420 commit, I will try it ASAP
and report the results here.
[Sun Sep 18 01:42:52.285253 2016] [wsgi:error] [pid 6470:tid 139771996858112]
Traceback (most recent call last):
[Sun Sep 18 01:42:52.285256 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../openstack_dashboard/dashboards/project/instances/tables.py",
line 389, in allowed
[Sun Sep 18 01:42:52.285259 2016] [wsgi:error] [pid 6470:tid 139771996858112]
limits = api.nova.tenant_absolute_limits(request, reserved=True)
[Sun Sep 18 01:42:52.285261 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../openstack_dashboard/api/nova.py",
line 949, in tenant_absolute_limits
[Sun Sep 18 01:42:52.285264 2016] [wsgi:error] [pid 6470:tid 139771996858112]
limits = novaclient(request).limits.get(reserved=reserved).absolute
[Sun Sep 18 01:42:52.285266 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../novaclient/v2/limits.py",
line 100, in get
[Sun Sep 18 01:42:52.285269 2016] [wsgi:error] [pid 6470:tid 139771996858112]
return self._get("/limits%s" % query_string, "limits")
[Sun Sep 18 01:42:52.285271 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../novaclient/base.py",
line 346, in _get
[Sun Sep 18 01:42:52.285273 2016] [wsgi:error] [pid 6470:tid 139771996858112]
resp, body = self.api.client.get(url)
[Sun Sep 18 01:42:52.285276 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../novaclient/client.py",
line 480, in get
[Sun Sep 18 01:42:52.285278 2016] [wsgi:error] [pid 6470:tid 139771996858112]
return self._cs_request(url, 'GET', **kwargs)
[Sun Sep 18 01:42:52.285280 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../novaclient/client.py",
line 458, in _cs_request
[Sun Sep 18 01:42:52.285282 2016] [wsgi:error] [pid 6470:tid 139771996858112]
resp, body = self._time_request(url, method, **kwargs)
[Sun Sep 18 01:42:52.285285 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../novaclient/client.py",
line 431, in _time_request
[Sun Sep 18 01:42:52.285287 2016] [wsgi:error] [pid 6470:tid 139771996858112]
resp, body = self.request(url, method, **kwargs)
[Sun Sep 18 01:42:52.285289 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../novaclient/client.py",
line 396, in request
[Sun Sep 18 01:42:52.285299 2016] [wsgi:error] [pid 6470:tid 139771996858112]
**kwargs)
[Sun Sep 18 01:42:52.285303 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../requests/api.py",
line 56, in request
[Sun Sep 18 01:42:52.285307 2016] [wsgi:error] [pid 6470:tid 139771996858112]
return session.request(method=method, url=url, **kwargs)
[Sun Sep 18 01:42:52.285310 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../requests/sessions.py",
line 475, in request
[Sun Sep 18 01:42:52.285313 2016] [wsgi:error] [pid 6470:tid 139771996858112]
resp = self.send(prep, **send_kwargs)
[Sun Sep 18 01:42:52.285317 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../requests/sessions.py",
line 596, in send
[Sun Sep 18 01:42:52.285320 2016] [wsgi:error] [pid 6470:tid 139771996858112]
r = adapter.send(request, **kwargs)
[Sun Sep 18 01:42:52.285324 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../requests/adapters.py",
line 497, in send
[Sun Sep 18 01:42:52.285328 2016] [wsgi:error] [pid 6470:tid 139771996858112]
raise SSLError(e, request=request)
[Sun Sep 18 01:42:52.285330 2016] [wsgi:error] [pid 6470:tid 139771996858112]
SSLError: ('bad handshake: SysCallError(0, None)',)
[Sun Sep 18 01:42:52.308704 2016] [wsgi:error] [pid 6470:tid 139771996858112]
Unable to retrieve project list.
[Sun Sep 18 01:42:52.308729 2016] [wsgi:error] [pid 6470:tid 139771996858112]
Traceback (most recent call last):
[Sun Sep 18 01:42:52.308734 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../openstack_auth/user.py",
line 318, in authorized_tenants
[Sun Sep 18 01:42:52.308738 2016] [wsgi:error] [pid 6470:tid 139771996858112]
is_federated=self.is_federated)
[Sun Sep 18 01:42:52.308742 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../openstack_auth/utils.py",
line 365, in get_project_list
[Sun Sep 18 01:42:52.308747 2016] [wsgi:error] [pid 6470:tid 139771996858112]
projects = client.projects.list(user=kwargs.get('user_id'))
[Sun Sep 18 01:42:52.308751 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../positional/__init__.py",
line 101, in inner
[Sun Sep 18 01:42:52.308755 2016] [wsgi:error] [pid 6470:tid 139771996858112]
return wrapped(*args, **kwargs)
[Sun Sep 18 01:42:52.308759 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../keystoneclient/v3/projects.py",
line 119, in list
[Sun Sep 18 01:42:52.308763 2016] [wsgi:error] [pid 6470:tid 139771996858112]
**kwargs)
[Sun Sep 18 01:42:52.308767 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../keystoneclient/base.py",
line 74, in func
[Sun Sep 18 01:42:52.308771 2016] [wsgi:error] [pid 6470:tid 139771996858112]
return f(*args, **new_kwargs)
[Sun Sep 18 01:42:52.308774 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../keystoneclient/base.py",
line 386, in list
[Sun Sep 18 01:42:52.308778 2016] [wsgi:error] [pid 6470:tid 139771996858112]
self.collection_key)
[Sun Sep 18 01:42:52.308782 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../keystoneclient/base.py",
line 124, in _list
[Sun Sep 18 01:42:52.308797 2016] [wsgi:error] [pid 6470:tid 139771996858112]
resp, body = self.client.get(url, **kwargs)
[Sun Sep 18 01:42:52.308802 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../keystoneauth1/adapter.py",
line 187, in get
[Sun Sep 18 01:42:52.308806 2016] [wsgi:error] [pid 6470:tid 139771996858112]
return self.request(url, 'GET', **kwargs)
[Sun Sep 18 01:42:52.308810 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../keystoneauth1/adapter.py",
line 344, in request
[Sun Sep 18 01:42:52.308814 2016] [wsgi:error] [pid 6470:tid 139771996858112]
resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)
[Sun Sep 18 01:42:52.308817 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../keystoneauth1/adapter.py",
line 112, in request
[Sun Sep 18 01:42:52.308822 2016] [wsgi:error] [pid 6470:tid 139771996858112]
return self.session.request(url, method, **kwargs)
[Sun Sep 18 01:42:52.308825 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../positional/__init__.py",
line 101, in inner
[Sun Sep 18 01:42:52.308829 2016] [wsgi:error] [pid 6470:tid 139771996858112]
return wrapped(*args, **kwargs)
[Sun Sep 18 01:42:52.308833 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../keystoneauth1/session.py",
line 555, in request
[Sun Sep 18 01:42:52.308837 2016] [wsgi:error] [pid 6470:tid 139771996858112]
resp = send(**kwargs)
[Sun Sep 18 01:42:52.308840 2016] [wsgi:error] [pid 6470:tid 139771996858112]
File
"/openstack/venvs/horizon-14.0.0/lib/python2.7/site-packages/openstack_dashboard/wsgi/../../keystoneauth1/session.py",
line 593, in _send_request
[Sun Sep 18 01:42:52.308844 2016] [wsgi:error] [pid 6470:tid 139771996858112]
raise exceptions.SSLError(msg)
[Sun Sep 18 01:42:52.308848 2016] [wsgi:error] [pid 6470:tid 139771996858112]
SSLError: SSL exception connecting to
https://cloud.ufscar.br:5000/v3/users/f1a04d6876eb4252b78c01059ca557bd/projects:
('bad handshake: SysCallError(0, None)',)
To manage notifications about this bug go to:
https://bugs.launchpad.net/horizon/+bug/1624791/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
