@leehom: Bug skimming result =================== Juno has reached end of life (EOL) [1] and won't get any updates anymore. That's why I close this bug as "Invalid".
If you observe the issue(s) in one or more of the supported releases (kilo, liberty, mitaka=master), then reopen the bug report and provide more information with template [2] to allow an analysis of the issue. If you have questions, I'm available in the IRC channel #openstack-nova under the name "markus_z". References ========== [1] http://docs.openstack.org/releases/ [2] https://wiki.openstack.org/wiki/Nova/BugsTeam/BugReportTemplate ** Changed in: nova Status: New => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1534066 Title: Resize failed when instance get large Status in OpenStack Compute (nova): Invalid Bug description: I'm using OpenStack Juno $ rpm -qa |grep nova python-nova-2014.2.2-1.el7.noarch openstack-nova-scheduler-2014.2.2-1.el7.noarch python-novaclient-2.20.0-1.el7.centos.noarch openstack-nova-common-2014.2.2-1.el7.noarch openstack-nova-api-2014.2.2-1.el7.noarch openstack-nova-cert-2014.2.2-1.el7.noarch openstack-nova-console-2014.2.2-1.el7.noarch openstack-nova-conductor-2014.2.2-1.el7.noarch openstack-nova-novncproxy-2014.2.2-1.el7.noarch There has two problems Problem1: Token not expired, but somehow deleted which cause /usr/lib/python2.7/site-packages/neutronclient/client.py:do_request() failed with exceptions.Unauthorized def do_request(self, url, method, **kwargs): self.authenticate_and_fetch_endpoint_url() # Perform the request once. If we get a 401 back then it # might be because the auth token expired, so try to # re-authenticate and try again. If it still fails, bail. try: kwargs.setdefault('headers', {}) if self.auth_token is None: self.auth_token = "" kwargs['headers']['X-Auth-Token'] = self.auth_token resp, body = self._cs_request(self.endpoint_url + url, method, **kwargs) return resp, body except exceptions.Unauthorized: self.authenticate() kwargs.setdefault('headers', {}) kwargs['headers']['X-Auth-Token'] = self.auth_token resp, body = self._cs_request( self.endpoint_url + url, method, **kwargs) return resp, body Problem2: In except branch, self.authenticate() failed with exceptions.Unauthorized which is a duplicate of https://bugs.launchpad.net/nova/+bug/1418529 But there has no fix for juno. Here is log found from nova-compute ########### Log for problem1 2016-01-07 16:08:42.396 1334 DEBUG neutronclient.client [req-7d95c82d-b26a-4584-851b-95de5c99f17c ] REQ: curl -i https://hf1-neutron.qa.webex.com:443/v2.0/extensions.json -X GET -H "X-Auth-Token: 119b7628442f4e19b4bd1041d05c2afa" -H "User-Agent: python-neutronclient" http_log_req /usr/lib/python2.7/site-packages/neutronclient/common/utils.py:140 2016-01-07 16:08:42.473 1334 DEBUG neutronclient.client [req-7d95c82d-b26a-4584-851b-95de5c99f17c ] RESP:401 {'content-length': '23', 'www-authenticate': "Keystone uri='https://hf1-keystone-srv.qa.webex.com:443/v2.0'", 'connection': 'keep-alive', 'date': 'Thu, 07 Jan 2016 16:08:42 GMT', 'content-type': 'text/plain', 'x-openstack-request-id': 'req-36035958-9e5d-44db-a12a-0ed0764a7d0e'} Authentication required http_log_resp /usr/lib/python2.7/site-packages/neutronclient/common/utils.py:149 ########### Log for problem2 2016-01-07 16:08:42.474 1334 ERROR nova.compute.manager [req-7d95c82d-b26a-4584-851b-95de5c99f17c None] [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] Setting instance vm_state to ERROR 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] Traceback (most recent call last): 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3904, in finish_resize 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] disk_info, image) 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3843, in _finish_resize 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] migration_p) 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 1241, in migrate_instance_finish 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] if not self._has_port_binding_extension(context, refresh_cache=True): 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 471, in _has_port_binding_extension 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] self._refresh_neutron_extensions_cache(context) 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 463, in _refresh_neutron_extensions_cache 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] extensions_list = neutron.list_extensions()['extensions'] 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 98, in with_params 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] ret = self.function(instance, *args, **kwargs) 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 300, in list_extensions 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] return self.get(self.extensions_path, params=_params) 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1320, in get 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] headers=headers, params=params) 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1297, in retry_request 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] headers=headers, params=params) 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1240, in do_request 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] content_type=self.content_type()) 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 183, in do_request 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] self.authenticate() 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 247, in authenticate 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] self._authenticate_keystone() 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 220, in _authenticate_keystone 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] raise exceptions.NoAuthURLProvided() 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] NoAuthURLProvided: auth_url was not provided to the Neutron client 2016-01-07 16:08:42.474 1334 TRACE nova.compute.manager [instance: fc8d35d5-874d-4c09-9f23-9bf7587b2127] Here is the log found from keystone ########## Log for problem1 2016-01-07 05:16:33.697 21392 DEBUG keystone.common.wsgi [-] arg_dict: {'token_id': u'119b7628442f4e19b4bd1041d05c2afa'} __call__ /usr/lib/python2.7/site-packages/keystone/common/wsgi.py:191 2016-01-07 05:16:33.698 21392 DEBUG keystone.common.controller [-] RBAC: Authorizing identity:validate_token(token_id=119b7628442f4e19b4bd1041d05c2afa) _build_policy_check_credentials /usr/lib/python2.7/site-packages/keystone/common/controller.py:55 2016-01-07 05:16:33.721 21392 INFO eventlet.wsgi.server [-] 10.224.241.140,10.224.236.93 - - [07/Jan/2016 05:16:33] "GET /v2.0/tokens/119b7628442f4e19b4bd1041d05c2afa HTTP/1.1" 200 2157 0.040029 2016-01-07 05:22:00.632 21393 DEBUG keystone.common.wsgi [-] arg_dict: {'token_id': u'119b7628442f4e19b4bd1041d05c2afa'} __call__ /usr/lib/python2.7/site-packages/keystone/common/wsgi.py:191 2016-01-07 05:22:00.633 21393 DEBUG keystone.common.controller [-] RBAC: Authorizing identity:validate_token(token_id=119b7628442f4e19b4bd1041d05c2afa) _build_policy_check_credentials /usr/lib/python2.7/site-packages/keystone/common/controller.py:55 2016-01-07 05:22:00.648 21393 INFO eventlet.wsgi.server [-] 10.224.241.139,10.224.236.93 - - [07/Jan/2016 05:22:00] "GET /v2.0/tokens/119b7628442f4e19b4bd1041d05c2afa HTTP/1.1" 200 2157 0.034250 2016-01-07 06:19:05.034 21410 DEBUG keystone.common.wsgi [-] arg_dict: {'token_id': u'119b7628442f4e19b4bd1041d05c2afa'} __call__ /usr/lib/python2.7/site-packages/keystone/common/wsgi.py:191 ########## I think this part is the key, but I do not understand where and why need to delete this token before it expire 2016-01-07 06:19:05.069 21410 INFO eventlet.wsgi.server [-] 10.224.241.139,10.224.236.93 - - [07/Jan/2016 06:19:05] "DELETE /v2.0/tokens/119b7628442f4e19b4bd1041d05c2afa HTTP/1.1" 204 127 0.049150 And here is found token from database ########## Log for problem1 | id | expires | extra | valid | trust_id | user_id | | 119b7628442f4e19b4bd1041d05c2afa | 2016-01-08 04:47:23 | {"bind": null, "token_data": {"access": {"token": {"issued_at": "2016-01-07T04:48:10.638860", "expires": "2016-01-08T04:47:23Z", ... | 0 | NULL | e35c77abe1384bf2a1f50363d870b44f | To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1534066/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
