Yeah, looks like Michael even realized it was going to be a breaking change:
https://github.com/openstack/nova/blob/743d5efccaa99e3b4873831a8f43c216a31c7113/nova/virt/libvirt/driver.py#L2766 ** Tags added: ceph configdrive libvirt ** Changed in: nova Status: New => Confirmed ** Changed in: nova Importance: Undecided => High ** Also affects: nova/liberty Importance: Undecided Status: New ** Also affects: nova/mitaka Importance: Undecided Status: New ** Summary changed: - nova kilo liberty ceph configdrive upgrade + nova kilo->liberty ceph configdrive upgrade fails ** Changed in: nova/liberty Status: New => Confirmed ** Changed in: nova/mitaka Status: New => Confirmed -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1582684 Title: nova kilo->liberty ceph configdrive upgrade fails Status in OpenStack Compute (nova): Confirmed Status in OpenStack Compute (nova) liberty series: Confirmed Status in OpenStack Compute (nova) mitaka series: Confirmed Bug description: Using CEPH RBD as our ephemeral drive led to an issue when upgrading from Kilo to Liberty. Our environment has "force_config_drive = True". In Icehouse, Juno, and Kilo, this uses an ISO 9660 image created in /var/lib/nova/instances/$UUID/disk.config However, in Liberty, if using CEPH RBD for ephemeral, there is a switch to putting this in rbe like this: rbd:instances/${UUID}_disk.config While this works GREAT for new VMs, it is problematic with existing VMs as not all transition states were considered. In particular, if you do a nova stop $UUID followed by a nova start $UUID you will find your instance still in the stopped state. There is something in the start code that ASSUMES that the new rbd format will be in place (but it doesn't actually create it.) There is a work around if you find instances in that state, simply cold migrate them with nova migrate $UUID which redoes the config.drive plumbing and creates the rbd:instances/${UUID}_disk.config Our permanent work around has been to prepopulate the rbd via a script though getting this bug fixed would be much better. Liberty is a stable release and this is a loss of service type of bug so should get fixed. Not clear if this is also an issue (likely so) in Mitaka/Newton as we haven't got an environment yet to test it, but presumably with long running VMs from early config drive, it would also exist in Mitaka. Specifics: Liberty Nova nova:12.0.2-38-g7bc3355.13.1b76006 CEPH: 0.94.6-1trusty Host OS: Ubuntu Trusty To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1582684/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
