On Tue, 2014-03-04 at 16:31 +0100, Luke Gorrie wrote: > Option 1: Should I debug the CI implementation we developed and tested > back in December, which was not based on Tempest but rather on our > private integration test method that we used in the Havana cycle? The > debugging that is needed is more defensive programming in our > automated attempt to setup OpenStack for test -- so that if the > installation fails for some unexpected reason we don't vote based on > that. > > Option 2: Should I start over with a standard Tempest test insead? If > so, what's the best method to set it up (yours? Arista's? another?), > and how do I know when that method is sufficiently debugged that it's > time to start?
Although I recognize you and your team have put in a substantial amount of time in debugging your custom setup, I would advise dropping the custom CI setup and going with a method that specifically uses the upstream openstack-dev/devstack and openstack-infra/devstack-gate projects. The reason is because these two projects are well supported by the upstream Infrastructure team. devstack will allow you to set up a complete OpenStack environment that matches upstream -- with the exception of using the Tailf-NCS ML2 plugin instead of the default plugin. devstack-gate will provide you the git checkout plumbing that will populate the source directories for the OpenStack projects that devstack uses to build its OpenStack environment. I'd recommend using my os-ext-testing repository (which is mostly just a couple shell scripts and documentation that uses the upstream Puppet modules to install and configure Jenkins, Zuul, Jenkins Job Builder, Gearman, devstack-gate/nodepool scripts on a master and slave node). > I was on the 3rd party testing meeting last night (as 'lukego') and > your recommendation for me was to hold off for a week or so and then > try your method after your next update. That sounds totally fine to me > in principle. However, this will mean that I don't have a mature test > procedure in place by March 14th, and I'm concerned that there may be > bad consequences on this. This date was mentioned as a deadline in the > Neutron meeting last night, but I don't actually understand what the > consequence of non-compliance is for established drivers such as this > one. I'm not going to step on Mark McClain's toes regarding policy for drivers in the Neutron code tree; Mark, please chime in here. I mentioned waiting about a week because, after discussions with the upstream Infrastructure team yesterday, it became clear that putting a nodepool manager in place to spin up *single-use devstack slave nodes* for running Tempest tests is going to be necessary. I had previously thought that it was possible to reset a Devstack environment to a clean state (thus being able to re-use the slave Jenkins node for >1 test run). However, so much is changed on the slave node during a Tempest run (and by devstack itself), that the only way to truly ensure a clean test environment is to have a brand new devstack slave node created/launched for each test run. Nodepool is the piece of software that manages a pool of these devstack slave nodes, and it will take me about a week to complete a new article and testing on the os-ext-testing repository for integrating and installing nodepool properly. Best, -jay _______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
