** Changed in: glance
Status: In Progress => Won't Fix
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/1914665
Title:
Cinder Multistore job hits Cinder Quota error
Status in Glance:
Won't Fix
Bug description:
Noticed during a cinder multistore test run, we hit a quota not found
error. It looks like we don't handle this well, which causes nova to
see a 503: Proxy Error. I dunno if there's anything better can do than
raise a 5xx, but we should probably explain in the error what happened
when we know, as we clearly do here.
From this:
https://cbff25b854b00bc0ff99-8ce5690b0835baabd00baac02d43f418.ssl.cf5.rackcdn.com/770629/5/check
/glance-multistore-cinder-
import/7c71502/controller/logs/screen-g-api.txt
this log text (see the end):
Feb 04 21:07:13.368998 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi Traceback (most recent
call last):
Feb 04 21:07:13.368998 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/opt/stack/glance/glance/common/wsgi.py", line 1347, in __call__
Feb 04 21:07:13.368998 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi action_result =
self.dispatch(self.controller, action,
Feb 04 21:07:13.368998 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/opt/stack/glance/glance/common/wsgi.py", line 1391, in dispatch
Feb 04 21:07:13.368998 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi return
method(*args, **kwargs)
Feb 04 21:07:13.368998 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/opt/stack/glance/glance/common/utils.py", line 416, in wrapped
Feb 04 21:07:13.368998 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi return func(self,
req, *args, **kwargs)
Feb 04 21:07:13.368998 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/opt/stack/glance/glance/api/v2/image_data.py", line 299, in upload
Feb 04 21:07:13.368998 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi
self._restore(image_repo, image)
Feb 04 21:07:13.368998 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 220, in
__exit__
Feb 04 21:07:13.368998 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi self.force_reraise()
Feb 04 21:07:13.368998 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
Feb 04 21:07:13.368998 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi
six.reraise(self.type_, self.value, self.tb)
Feb 04 21:07:13.368998 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/usr/local/lib/python3.8/dist-packages/six.py", line 703, in reraise
Feb 04 21:07:13.368998 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi raise value
Feb 04 21:07:13.370209 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/opt/stack/glance/glance/api/v2/image_data.py", line 164, in upload
Feb 04 21:07:13.370209 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi
image.set_data(data, size, backend=backend)
Feb 04 21:07:13.370209 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/opt/stack/glance/glance/domain/proxy.py", line 208, in set_data
Feb 04 21:07:13.370209 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi
self.base.set_data(data, size, backend=backend, set_active=set_active)
Feb 04 21:07:13.370209 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/opt/stack/glance/glance/notifier.py", line 501, in set_data
Feb 04 21:07:13.370209 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi
_send_notification(notify_error, 'image.upload', msg)
Feb 04 21:07:13.370209 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 220, in
__exit__
Feb 04 21:07:13.370209 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi self.force_reraise()
Feb 04 21:07:13.370209 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
Feb 04 21:07:13.370209 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi
six.reraise(self.type_, self.value, self.tb)
Feb 04 21:07:13.370209 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/usr/local/lib/python3.8/dist-packages/six.py", line 703, in reraise
Feb 04 21:07:13.370209 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi raise value
Feb 04 21:07:13.370209 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/opt/stack/glance/glance/notifier.py", line 447, in set_data
Feb 04 21:07:13.370209 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi
self.repo.set_data(data, size, backend=backend,
Feb 04 21:07:13.370209 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/opt/stack/glance/glance/api/policy.py", line 206, in set_data
Feb 04 21:07:13.370209 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi return
self.image.set_data(*args, **kwargs)
Feb 04 21:07:13.370209 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/opt/stack/glance/glance/quota/__init__.py", line 322, in set_data
Feb 04 21:07:13.370209 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi
self.image.set_data(data, size=size, backend=backend,
Feb 04 21:07:13.370209 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/opt/stack/glance/glance/location.py", line 567, in set_data
Feb 04 21:07:13.370209 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi
self._upload_to_store(data, verifier, backend, size)
Feb 04 21:07:13.371474 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/opt/stack/glance/glance/location.py", line 458, in _upload_to_store
Feb 04 21:07:13.371474 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi multihash,
loc_meta) = self.store_api.add_with_multihash(
Feb 04 21:07:13.371474 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/usr/local/lib/python3.8/dist-packages/glance_store/multi_backend.py", line
398, in add_with_multihash
Feb 04 21:07:13.371474 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi return
store_add_to_backend_with_multihash(
Feb 04 21:07:13.371474 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/usr/local/lib/python3.8/dist-packages/glance_store/multi_backend.py", line
480, in store_add_to_backend_with_multihash
Feb 04 21:07:13.371474 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi (location, size,
checksum, multihash, metadata) = store.add(
Feb 04 21:07:13.371474 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/usr/local/lib/python3.8/dist-packages/glance_store/driver.py", line 279, in
add_adapter
Feb 04 21:07:13.371474 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi metadata_dict) =
store_add_fun(*args, **kwargs)
Feb 04 21:07:13.371474 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/usr/local/lib/python3.8/dist-packages/glance_store/capabilities.py", line
176, in op_checker
Feb 04 21:07:13.371474 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi return
store_op_fun(store, *args, **kwargs)
Feb 04 21:07:13.371474 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/usr/local/lib/python3.8/dist-packages/glance_store/_drivers/cinder.py", line
852, in add
Feb 04 21:07:13.371474 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi volume =
client.volumes.create(size_gb, name=name, metadata=metadata,
Feb 04 21:07:13.371474 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes.py", line 125,
in create
Feb 04 21:07:13.371474 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi return
self._create('/volumes', body, 'volume')
Feb 04 21:07:13.371474 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 302, in
_create
Feb 04 21:07:13.371474 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi resp, body =
self.api.client.post(url, body=body)
Feb 04 21:07:13.371474 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 485, in
post
Feb 04 21:07:13.372519 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi return
self._cs_request(url, 'POST', **kwargs)
Feb 04 21:07:13.372519 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 438, in
_cs_request
Feb 04 21:07:13.372519 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi resp, body =
self.request(url, method, **kwargs)
Feb 04 21:07:13.372519 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi File
"/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 420, in
request
Feb 04 21:07:13.372519 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi raise
exceptions.from_response(resp, body)
Feb 04 21:07:13.372519 ubuntu-focal-limestone-regionone-0022873642
devstack@g-api.service[93292]: ERROR glance.common.wsgi
cinderclient.exceptions.OverLimit: VolumeLimitExceeded: Maximum number of
volumes allowed (10) exceeded for quota 'volumes'. (HTTP 413) (Request-ID:
req-5a3e84d0-70fb-425c-b8b2-fdddc14b7bf1)
To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/1914665/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
