Reviewed: https://review.openstack.org/294513
Committed:
https://git.openstack.org/cgit/openstack/nova/commit/?id=d8e695cb900ad71996e6cf970894bc2e3f2df8b4
Submitter: Jenkins
Branch: master
commit d8e695cb900ad71996e6cf970894bc2e3f2df8b4
Author: Prateek Arora <par...@redhat.com>
Date: Fri Mar 18 06:46:43 2016 -0400
Delete traces of in-progress snapshot on VM being deleted
When user tries to create snapshot of instance and at the same time
if another request tries to delete the instance, at that time image
goes in saving status for forever because of race between instance
delete and snapshot requests.
Caught exceptions(InstanceNotFound and UnexpectedDeletingTaskStateError)
in except block and deleting the image which got stuck in saving state.
Closes-Bug: #1555065
Change-Id: If0b918dc951030e6b6ffba147443225e0e4a370a
** Changed in: nova
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1555065
Title:
Image goes to saving state when we delete instance just after taking
snapshot and remain the state forever
Status in OpenStack Compute (nova):
Fix Released
Bug description:
Steps to reproduce
1) nova image-create test test-img & nova delete test
where test is the name of the instance
I get the following message
[stack@localhost compute]$ nova image-create test test-img & nova delete test
[1] 2506
ERROR (ClientException): Unexpected API Error. Please report this at
http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'nova.exception.UnexpectedDeletingTaskStateError'> (HTTP 500)
(Request-ID: req-f699c746-0b29-4d84-b258-c8233d5b7a42)
Request to delete server test has been accepted.
[1]+ Exit 1 nova image-create test test-img
In the nova api logs i can see the following stacktrace
'c59f52ce-b51f-433e-81b0-099e3b65b0a3',
'89ba4861-99e2-4376-8015-3f5ac02538d0']^[[00m ^[[00;33mfrom (pid=12399) reserve
/opt/stack/nova/nova/quota.py:1345^[[00m
2016-03-09 02:04:25.304 ^[[01;31mERROR nova.api.openstack.extensions
[^[[01;36mreq-f699c746-0b29-4d84-b258-c8233d5b7a42 ^[[00;36madmin
admin^[[01;31m] ^[[01;35m^[[01;31mUnexpected exception in API method^[[00m
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00mTraceback (most recent call last):
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m File "/opt/stack/nova/nova/api/openstack/extensions.py", line
478, in wrapped
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m return f(*args, **kwargs)
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m File "/opt/stack/nova/nova/api/openstack/common.py", line 391,
in inner
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m return f(*args, **kwargs)
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m File "/opt/stack/nova/nova/api/validation/__init__.py", line
73, in wrapper
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m return func(*args, **kwargs)
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m File "/opt/stack/nova/nova/api/validation/__init__.py", line
73, in wrapper
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m return func(*args, **kwargs)
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m File "/opt/stack/nova/nova/api/openstack/compute/servers.py",
line 1113, in _action_create_image
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m extra_properties=metadata)
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m File "/opt/stack/nova/nova/compute/api.py", line 169, in
wrapped
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m return func(self, context, target, *args, **kwargs)
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m File "/opt/stack/nova/nova/compute/api.py", line 186, in
_wrapped
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m return fn(self, context, instance, *args, **kwargs)
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m File "/opt/stack/nova/nova/compute/api.py", line 139, in inner
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m return f(self, context, instance, *args, **kw)
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m File "/opt/stack/nova/nova/compute/api.py", line 2238, in
snapshot
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m instance.save(expected_task_state=[None])
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m File
"/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 223, in
wrapper
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m return fn(self, *args, **kwargs)
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m File "/opt/stack/nova/nova/objects/instance.py", line 694, in
save
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m columns_to_join=_expected_cols(expected_attrs))
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m File "/opt/stack/nova/nova/db/api.py", line 805, in
instance_update_and_get_original
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m expected=expected)
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 229, in
wrapper
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m return f(*args, **kwargs)
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line
148, in wrapper
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m ectxt.value = e.inner_exc
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m File
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m self.force_reraise()
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m File
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in
force_reraise
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m six.reraise(self.type_, self.value, self.tb)
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line
138, in wrapper
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m return f(*args, **kwargs)
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 300, in
wrapped
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m return f(context, *args, **kwargs)
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 2707,
in instance_update_and_get_original
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m context, instance_uuid, values, expected,
original=instance_ref))
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 2843,
in _instance_update
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m raise exc(**exc_props)
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00mUnexpectedDeletingTaskStateError: Conflict updating instance
74940cbc-3947-434c-93ce-9489faa78073. Expected: {'task_state': [None]}. Actual:
{'task_state': u'deleting'}
^[[01;31m2016-03-09 02:04:25.304 TRACE nova.api.openstack.extensions
^[[01;35m^[[00m
2016-03-09 02:04:25.311 ^[[00;36mINFO nova.api.openstack.wsgi
[^[[01;36mreq-f699c746-0b29-4d84-b258-c8233d5b7a42 ^[[00;36madmin
admin^[[00;36m] ^[[01;35m^[[00;36mHTTP exception thrown: Unexpected API Error.
Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API
log if possible.
<class 'nova.exception.UnexpectedDeletingTaskStateError'>^[[00m
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1555065/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
