On 04/16/2014 11:48 AM, Jaume Devesa wrote:
Hi Sean,
for what I understood, we will need a new feature flag for each new
feature, and a feature flag (default to false) for each deprecated
one. My concern is: since the goal is make tempest a confident tool to
test any installation and not and 'tempest.conf' will not be
auto-generated from any tool as devstack does, wouldn't be too hard to
prepare a tempest.conf file with so many feature flags to enable and
disable?
If we go down this route, and I think we should, we probably need to
accept that it will be hard for users to manually configure
tempest.conf. Tempest configuration would have to be done by whatever
installation technology was used, as devstack does, or by
auto-discovery. That implies that the presence of new features should be
discoverable through the api which is a good idea anyway. Of course some
one could configure it manually but IMO that is not desirable even with
where we are now.
Maybe I am simplifying too much, but wouldn't be enough with a pair of
functions decorators like
@new
@deprecated
Then, in tempest.conf it could be a flag to say which OpenStack
installation are you testing:
installation = [icehouse|juno]
if you choose Juno, tests with @new decorator will be executed and
tests with @deprecated will be skipped.
If you choose Icehouse, tests with @new decorator will be skipped, and
tests with @deprecated will be executed
I am missing some obvious case that make this approach a nonsense?
There are two problems with this. First, some folks are chasing master
for their deployments and some do not deploy all the features that are
set up by devstack. In both cases, it is not possible to identify what
can be tested with a simple name that corresponds to a stable release.
Second, what happens when we move on to K? The meaning of "new" would
have to change while retaining its old meaning as well which won't work.
I think Sean spelled out the important scenarios.
-David
Regards,
jaume
On 14 April 2014 15:21, Sean Dague <s...@dague.net
<mailto:s...@dague.net>> wrote:
As we're coming up on the stable/icehouse release the QA team is
looking
pretty positive at no longer branching Tempest. The QA Spec draft for
this is here -
http://docs-draft.openstack.org/77/86577/2/check/gate-qa-specs-docs/3f84796/doc/build/html/specs/branchless-tempest.html
and hopefully address a lot of the questions we've seen so far.
Additional comments are welcome on the review -
https://review.openstack.org/#/c/86577/
or as responses on this ML thread.
-Sean
--
Sean Dague
Samsung Research America
s...@dague.net <mailto:s...@dague.net> / sean.da...@samsung.com
<mailto:sean.da...@samsung.com>
http://dague.net
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
<mailto:OpenStack-dev@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
