John Griffith wrote: > So we've talked about this a bit and had a number of ideas regarding > how to test and show compatibility for third-party drivers in Cinder. > This has been an eye opening experience (the number of folks that have > NEVER run tempest before, as well as the problems uncovered now that > they're trying it). > > I'm even more convinced now that having vendors run these tests is a > good thing and should be required. That being said there's a ton of > push back from my proposal to require that results from a successful > run of the tempest tests to accompany any new drivers submitted to > Cinder.
Could you describe the nature of the pushback ? Is it that the tests are too deep and reject valid drivers ? Is it that it's deemed unfair to block new drivers while the existing ones aren't better ? Is it that it's difficult for them to run those tests and get a report ? Or is it because they care more about having their name covered in mainline and not so much about having the code working properly ? > The consensus from the Cinder community for now is that we'll > log a bug for each driver after I3, stating that it hasn't passed > certification tests. We'll then have a public record showing > drivers/vendors that haven't demonstrated functional compatibility, > and in order to close those bugs they'll be required to run the tests > and submit the results to the bug in Launchpad. > > So, this seems to be the approach we're taking for Icehouse at least, > it's far from ideal IMO, however I think it's still progress and it's > definitely exposed some issues with how drivers are currently > submitted to Cinder so those are positive things that we can learn > from and improve upon in future releases. > > To add some controversy and keep the original intent of having only > known tested and working drivers in the Cinder release, I am going to > propose that any driver that has not submitted successful functional > testing by RC1 that that driver be removed. I'd at least like to see > driver maintainers try... if the test fails a test or two that's > something that can be discussed, but it seems that until now most > drivers just flat out are not even being tested. I think there are multiple stages here. Stage 0: noone knows if drivers work Stage 1: we know the (potentially sad) state of the drivers that are in the release Stage 2: only drivers that pass tests are added, drivers that don't pass tests have a gap analysis and a plan to fix it Stage 3: drivers that fail tests are removed before release Stage 4: 3rd-party testing rigs must run tests on every change in order to stay in tree At the very minimum you should be at stage 1 for the Icehouse release, so I agree with your last paragraph. I'd recommend that you start the Juno cycle at stage 2 (for new drivers), and try to reach stage 3 for the end of the Juno release. -- Thierry Carrez (ttx) _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
