> In addition to seeking a spec-freeze-exception for 95025, I would also > like some clarification of the requirement to test this upgrade > path. Some nova-core folks have pointed out that they do not want to > accept the nova.virt.ironic driver until the upgrade path from > nova.virt.baremetal *has test coverage*, but what that means is not > clear to me. It's been suggested that we use grenade (I am pretty sure > Sean suggested this at the summit, and I wrote it into my spec proposal > soon thereafter). After looking into grenade, I don't think it is the > right tool to test with, and I'm concerned that no one pointed this out > sooner.
Grenade is our release test tool, so I think that, barring details, it's reasonable to use $GRENADE when talking about this sort of thing. I didn't realize that nova-bm doesn't work in devstack until you pointed it out in IRC last week. Since we're pretty good about requiring devstack support for new things like drivers, I would have expected nova-bm to work there, but obviously times were a bit different when that driver was merged. > Philosophically, this isn't an upgrade of one service from version X to > Y. It's a replacement of one nova driver with a completely different > driver. As I understand it, that's not what grenade is for. But maybe > I'm wrong on this, or maybe it's flexible. I think it's "start devstack on release X, validate, do some work, re-start devstack on release Y, validate". I'm not sure that it's ill-suited for this, but IANAGE. > I also have a technical objection: even if devstack can start and > properly configure nova.virt.baremteal (which I doubt, because it isn't > tested at all), it is going to fail the tempest/api/compute test suite > horribly. The baremetal driver never passed tempest, and never had > devstack-gate support. This matters because grenade uses tempest to > validate a stack pre- and post-upgrade. Therefore, since we know that > the old code is going to fail tempest, requiring grenade testing as a > precondition to accepting the ironic driver effectively means we need to > go develop the baremetal driver to a point it could pass tempest. I'm > going to assume no one is actually suggesting that, and instead believe > that none of us thought this through. > > (FWIW, Ironic doesn't pass the tempest/api/compute suite today, but > we're working hard on it.) Do the devstack exercises pass? We test things like cells today (/me hears sdague scream in the background), which don't pass tempest, using the exercises to make sure it's at least able to create an instance. > So, I'd like to ask for suggestions on what sort of upgrade testing is > reasonable here. I'll toss out two ideas: > - load some fake data into the nova_bm schema, run the upgrade scripts, > start ironic, issue some API queries, and see if the data's correct > - start devstack, load some real data into the nova_bm schema, run the > upgrade scripts, then try to deploy an instance with ironic These were my suggestions last week, so I'll own up to them now. Obviously I think that something using grenade that goes from a functional environment on release X to a functional environment on release Y is best. However, I of course don't think it makes sense to spend a ton of time getting nova-bm to pass tempest just so we can shoot it in the head. I'm not really sure what to do here. I think that we need an upgrade path, and that it needs to be tested. I don't think our users would appreciate us removing any other virt driver and replacing it with a new one, avoiding an upgrade path because "it's a different driver now". I also don't want to spend a bunch of time on nova-bm, which we have already neglected in our other test requirements (which is maybe part of the problem here). Assuming grenade can be flexible about what it runs against the old and new environments to determine "workyness", then I think the second option above is probably a pretty good level of assurance, given where we are right now. --Dan
signature.asc
Description: OpenPGP digital signature
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
