I think this clearly illustrates why it is so important for QA to be
done more in the open, and to post tempest blueprints for largish
projects like this. I have a theory that one important reason so many
organizations choose not to do this is that once they submit their tests
to the public repository (in this case tempest), they have to get every
change approved. I had the same issue when we submitted our stress
tests. There cannot be any advantage to doing qa in private. We need to
make submitting test code free of such barriers. More tests can only be
good.
I propose that we set up a 'contrib' part of tempest that is exempt from
core review. At some point tests can be moved into tempest proper
though, as a gating job, there is a limit to what can be there due to
runtime considerations. But we already have a lot of tests running every
night distributed across organizations. Why not post the code so others
can share and improve it? We would need to:
1. Set up the contrib directory
2. Establish some guidelines for submitting changes to other people's
contributions
3. Establish the nightly build for contributed tests, probably including
running some outside of jenkins
I am not sure what review flexibility is afforded by gerrit that would
help or hinder this effort.
Thoughts?
-David
On 8/16/2012 3:40 PM, Dan Smith wrote:
Hi Daryl,
DW> Sorry for any confusion.
I share at least half the blame here :/
DW> I've had an implementation for this working for awhile, but failed
DW> to port it in as I was working on getting other features
DW> out. Regardless of the implementation, I'm really happy to see this
DW> flexibility get intoTempest trunk. Since it seems you and your team
DW> have already invested significant work into this, I'll end my effort
DW> and focus on code reviews for the branches you've submitted.
Okay, well, thanks. I guess I should have done some more comprehensive
digging before I started down the path, so sorry for that.
Thanks in advance for the reviews. I know what I've got probably isn't
good enough to merge just yet, but am definitely looking forward to
hearing from some tempestuous folks :)
--
Mailing list: https://launchpad.net/~openstack-qa-team
Post to : openstack-qa-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack-qa-team
More help : https://help.launchpad.net/ListHelp
