** Changed in: nova
Status: In Progress => Won't Fix
** Changed in: nova
Status: Won't Fix => New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1640164
Title:
Rolling upgrade M to N: DBDeadlock Error when create instance during
sync database
Status in OpenStack Compute (nova):
New
Bug description:
I have 3 controller nodes running HA active/active, using KVM
hypervisor and Maria cluster as shared database. The system was
deployed by Devstack Mitaka version on virtual machines which was
created by virt-manager.
I have upgraded Keytone to N version, then I tried to Rolling Upgrade Nova
from M to N version folowed:
http://docs.openstack.org/developer/nova/upgrade.html#rolling-upgrade-process
The document said that:
Using the newly installed nova code, run the DB sync. (nova-manage db sync;
nova-manage api_db sync). These schema change operations should have minimal or
no effect on performance, and should not cause any operations to fail.
However, during the sync database, I cannot create the VM. Nova raise
that:
ERROR (ClientException): Unexpected API Error. Please report this at
http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_db.exception.DBDeadlock'> (HTTP 500) (Request-ID:
req-b5c82715-6306-4f6d-972b-3387015da12c)
Full log here: http://paste.openstack.org/show/588365/
After finishing the sync process, I can create VM as well.
==Reproduce==
# Controller1:
1. Stop all nova service, except nova api (n-api).
2. Upgrade source code:
$ cd /opt/stack/nova/
$ git checkout remotes/origin/stable/newton
$ git checkout -b stable/newton remotes/origin/stable/newton
$ git pull
$ sudo -E pip install -r requirements.txt --upgrade
3. Downgrade some packages dependency (because I used --upgrade as above)
$ sudo pip uninstall oslo.messaging
$ sudo pip uninstall kombu
$ sudo pip uninstall cffi
$ sudo -E pip install oslo.messaging==5.10.0
$ sudo -E pip install kombu==3.0.35
$ sudo -E pip install cffi==1.5.2
4. Update /etc/nova/nova.conf:
[upgrade_levels]
compute = auto
5. Sync DB
$ nova-manage db sync
$ nova-manage api_db sync
6. During the Sync DB, try to create VM, execute on controller 2 and 3 (not
concurrency):
$ nova boot --flavor m1.nano --image 21ffa33b-e9eb-43f4-aa73-ceb8f2cbc6fc
--nic net-name=net1 VM_test
ERROR (ClientException): Unexpected API Error. Please report this at
http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_db.exception.DBDeadlock'> (HTTP 500) (Request-ID:
req-b5c82715-6306-4f6d-972b-3387015da12c)
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1640164/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
