On Monday, November 25, 2013 4:37:29 PM, Russell Bryant wrote:
On 11/25/2013 05:19 PM, Matt Riedemann wrote:
I'll play devil's advocate here and ask this question before someone
else does. I'm assuming that the requirement of a 'full' tempest run
means running this [1]. Is that correct? It's just confusing sometimes
because there are other things in Tempest that aren't in the 'full' run,
like stress tests.
Assuming that's what 'full' means, it's running API, CLI, third party
(boto), and scenario tests. Does it make sense to require a nova virt
driver's CI to run API tests for keystone, heat and swift? Or couldn't
the nova virt driver CI be scoped down to just the compute API tests?
The argument against that is probably that the network/image/volume
tests may create instances using nova to do their API testing also. The
same would apply for the CLI tests since those are broken down by
service, i.e. why would I need to run keystone and ceilometer CLI tests
for a nova virt driver?
If nothing else, I think we could firm up the wording on the wiki a bit
around the requirements and what that means for scope.
[1] https://github.com/openstack/tempest/blob/master/tox.ini#L33
I think the short answer is, "whatever we're running against all Nova
changes in the gate".
Maybe a silly question, but is what is run against the check queue any
different from the gate queue?
I expect that for some drivers, a more specific configuration is going
to be needed to exclude tests for features not implemented in that
driver. That's fine.
Soon we also need to start solidifying criteria for what features *must*
be implemented in a driver. I think we've let some drivers in with far
too many features not supported. That's a separate issue from the CI
requirement, though.
--
Thanks,
Matt Riedemann
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
