Reviewed: https://review.openstack.org/367597
Committed:
https://git.openstack.org/cgit/openstack/nova/commit/?id=193e71bd30bbc6dd0a22b4e07d98e8a763ddda7b
Submitter: Jenkins
Branch: master
commit 193e71bd30bbc6dd0a22b4e07d98e8a763ddda7b
Author: Andrew Laski <and...@lascii.com>
Date: Thu Sep 8 15:42:36 2016 -0400
Handle ObjectActionError during cells instance delete
There was logic in place to handle an ObjectActionError being raised
when deleting an instance with cellsv1, however a second
instance.destroy() call was added which did not have the same logic
applied. This adds appropriate handling for that case.
Change-Id: I007f249f772dbb53cf24a9bf06d2962001cb8384
Closes-Bug: 1621574
** Changed in: nova
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1621574
Title:
Instance fails to delete while building in cells v1 due to
"ObjectActionError: Object action destroy failed because: host
changed"
Status in OpenStack Compute (nova):
Fix Released
Bug description:
Seeing this in a cells v1 job here:
http://logs.openstack.org/55/367455/1/check/gate-tempest-dsvm-
cells/1d8af83/logs/screen-n-api.txt.gz#_2016-09-08_15_59_02_553
2016-09-08 15:59:02.553 15924 ERROR nova.api.openstack.extensions
[req-0f23fed6-0aaf-461f-96d8-f4aa5055b208
tempest-ListServersNegativeTestJSON-1414534476
tempest-ListServersNegativeTestJSON-1414534476] Unexpected exception in API
method
2016-09-08 15:59:02.553 15924 ERROR nova.api.openstack.extensions Traceback
(most recent call last):
2016-09-08 15:59:02.553 15924 ERROR nova.api.openstack.extensions File
"/opt/stack/new/nova/nova/api/openstack/extensions.py", line 338, in wrapped
2016-09-08 15:59:02.553 15924 ERROR nova.api.openstack.extensions return
f(*args, **kwargs)
2016-09-08 15:59:02.553 15924 ERROR nova.api.openstack.extensions File
"/opt/stack/new/nova/nova/api/openstack/compute/servers.py", line 915, in delete
2016-09-08 15:59:02.553 15924 ERROR nova.api.openstack.extensions
self._delete(req.environ['nova.context'], req, id)
2016-09-08 15:59:02.553 15924 ERROR nova.api.openstack.extensions File
"/opt/stack/new/nova/nova/api/openstack/compute/servers.py", line 767, in
_delete
2016-09-08 15:59:02.553 15924 ERROR nova.api.openstack.extensions
self.compute_api.delete(context, instance)
2016-09-08 15:59:02.553 15924 ERROR nova.api.openstack.extensions File
"/opt/stack/new/nova/nova/compute/cells_api.py", line 213, in delete
2016-09-08 15:59:02.553 15924 ERROR nova.api.openstack.extensions
self._handle_cell_delete(context, instance, 'delete')
2016-09-08 15:59:02.553 15924 ERROR nova.api.openstack.extensions File
"/opt/stack/new/nova/nova/compute/cells_api.py", line 224, in
_handle_cell_delete
2016-09-08 15:59:02.553 15924 ERROR nova.api.openstack.extensions if
self._delete_while_booting(context, instance):
2016-09-08 15:59:02.553 15924 ERROR nova.api.openstack.extensions File
"/opt/stack/new/nova/nova/compute/api.py", line 1648, in _delete_while_booting
2016-09-08 15:59:02.553 15924 ERROR nova.api.openstack.extensions
instance.destroy()
2016-09-08 15:59:02.553 15924 ERROR nova.api.openstack.extensions File
"/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line
226, in wrapper
2016-09-08 15:59:02.553 15924 ERROR nova.api.openstack.extensions return
fn(self, *args, **kwargs)
2016-09-08 15:59:02.553 15924 ERROR nova.api.openstack.extensions File
"/opt/stack/new/nova/nova/objects/instance.py", line 555, in destroy
2016-09-08 15:59:02.553 15924 ERROR nova.api.openstack.extensions
reason='host changed')
2016-09-08 15:59:02.553 15924 ERROR nova.api.openstack.extensions
ObjectActionError: Object action destroy failed because: host changed
2016-09-08 15:59:02.553 15924 ERROR nova.api.openstack.extensions
http://logs.openstack.org/55/367455/1/check/gate-tempest-dsvm-
cells/1d8af83/console.html#_2016-09-08_16_13_48_892443
logs.openstack.org/55/367455/1/check/gate-tempest-dsvm-
cells/1d8af83/logs/tempest.txt.gz#_2016-09-08_15_59_02_558
In this test it creates a server and then immediately deletes it
without waiting for it to be ACTIVE first.
I wonder if we're racing with any of the BuildRequest stuff that's new
in Newton.
http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22ObjectActionError%3A%20Object%20action%20destroy%20failed%20because%3A%20host%20changed%5C%22%20AND%20message%3A%5C%22_handle_cell_delete%5C%22%20AND%20tags%3A%5C%22screen-n-api.txt%5C%22&from=7d
10 hits in the last 7 days, check and gate, master (newton) only at
this point.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1621574/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
