> -----Original Message----- > From: Matthew Treinish [mailto:mtrein...@kortar.org] > Sent: Wednesday, June 18, 2014 10:33 PM > To: OpenStack Development Mailing List (not for usage questions) > Subject: Re: [openstack-dev] [qa] Clarification of policy for qa-specs around > adding new tests > > On Tue, Jun 17, 2014 at 01:45:55AM +0000, Kenichi Oomichi wrote: > > > > > -----Original Message----- > > > From: Matthew Treinish [mailto:mtrein...@kortar.org] > > > Sent: Monday, June 16, 2014 11:58 PM > > > To: OpenStack Development Mailing List (not for usage questions) > > > Subject: Re: [openstack-dev] [qa] Clarification of policy for qa-specs > > > around adding new tests > > > > > > On Mon, Jun 16, 2014 at 10:46:51AM -0400, David Kranz wrote: > > > > I have been reviewing some of these specs and sense a lack of clarity > > > > around > > > > what is expected. In the pre-qa-specs world we did not want tempest > > > > blueprints to be used by projects to track their tempest test > > > > submissions > > > > because the core review team did not want to have to spend a lot of time > > > > dealing with that. We said that each project could have one tempest > > > > blueprint that would point to some other place (project blueprints, > > > > spreadsheet, etherpad, etc.) that would track specific tests to be > > > > added. > > > > I'm not sure what aspect of the new qa-spec process would make us feel > > > > differently about this. Has this policy changed? We should spell out the > > > > expectation in any event. I will update the README when we have a > > > > conclusion. > > > > > > > > > > The policy has not changed. There should be 1 BP (or maybe 2 or 3 if they > > > want > > > to split the effort a bit more granularly for tracking different classes > > > of > > > tests, but still 1 BP series) for improving project tests. For individual > > > tests > > > part of a bigger effort should be tracked outside of the Tempest LP. IMO > > > after > > > it's approved the spec/BP for tracking test additions is only really > > > useful to > > > have a unified topic to use for review classification. > > > > +1 to use a single blueprint for adding new tests of each project. > > The unified topic of each project would be useful to get each project > > reviewers' effort on the Tempest tests reviews. > > To add new tests, do we need to have qa-specs, or is it OK to have > > blueprints only? > > > > So I've been asking all the new BPs for project testing being opened this > cycle > to have a spec too. My feeling is that we should only have one process for > doing > BPs/specs that way we get all the artifacts in the same place. It should also > hopefully get everyone more involved with the qa-specs workflow.
I see, thank you for clarifying it. > The specs for adding project test should be pretty simple, they just basically > need to outline what project is going to be tested, what types of tests are > going to be worked on, (API, CLI, etc..) and how the test development is > going > to be tracked. (etherpad, google doc, etc.) That is nice advice, it seems easy to review also :-) Thanks Ken'ichi Ohmichi _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
