On 08/12/15 15:31 -0500, Matthew Treinish wrote:
On Tue, Dec 08, 2015 at 07:25:50PM +0000, Flavio Percoco wrote:
Greetings,
I just reviewed a patch in tempest that proposed adding new tests for
the Glance's task API. While I believe it's awesome that folks in the
tempest team keep adding missing tests for projects, I think it'd be
better for the tempest team, the project's team and defcore if we'd
discuss these tests before they are worked on. This should help people
avoid wasting time.
I agree a tasks API test isn't really something we want in tempest. I
actually didn't realize that there was a proposed test for this. (I've
been a bit lax in my tempest reviews lately)
I believe these cases are rare but the benefits of discussing missing
tests across teams could also help prioritizing the work based on what
the teams goals are, what the defcore team needs are, etc.
So, I'd like to start improving this process by inviting folks from
the tempest team to join project's meeting whenever new tests are
going to be worked on.
So, TBH I don't think this will really scale too well. The team of people
dedicated to working on tempest is relatively small compared the size of
OpenStack. I think taking the reverse approach and having project teams
taking a more hands on approach to tempest tests is the only way we'll be
able to scale here.
Oh, I meant ocasionally. Whenever a missing test for an API is found,
it'd be easy enough for the implementer to sohw up at the meeting and
bring it up.
ML threads like this are actually a good approach on this front I think. It
raises awareness about a proposed test that has potential issues and discusses
it in an archived place.
This works for me too! :)
Cheers,
Flavio
I'd also like to invite PTLs (or anyone, really) from each team to go
through what's in tempest and what's missing and help this team
improve the test suite. Remember that these tests are also used by the
defcore team and they are not important just for the CI but have an
impact on other areas as well.
I'm doing the above for Glance and I can't stress enough how important
it is for projects to do the same
A very big +1, on these 2 paragraphs. Tempest is used for much more than gating
and making sure we have what projects view as important is covered is important.
Do teams have a different workflow/process to increase tests coverage
in tempest?
I'm not aware of anything going on like this now. At one point in the past as
part of tempest development we used to have a single tempest BP per project per
cycle that project teams would use to collect and track test development for a
cycle. But, I haven't seen that in at least a year, likely longer.
-Matt Treinish
--
@flaper87
Flavio Percoco
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
