On 25/07/14 11:18, Sean Dague wrote: > On 07/25/2014 10:01 AM, Steven Hardy wrote: >> On Wed, Jul 23, 2014 at 02:39:47PM -0700, James E. Blair wrote: >> <snip> >>> * Put the burden for a bunch of these tests back on the projects as >>> "functional" tests. Basically a custom devstack environment that a >>> project can create with a set of services that they minimally need >>> to do their job. These functional tests will live in the project >>> tree, not in Tempest, so can be atomically landed as part of the >>> project normal development process. >> +1 - FWIW I don't think the current process where we require tempest >> cores to review our project test cases is working well, so allowing >> projects to own their own tests will be a major improvement. >> >> In terms of how this works in practice, will the in-tree tests still be run >> via tempest, e.g will there be a (relatively) stable tempest api we can >> develop the tests against, as Angus has already mentioned? > No, not run by tempest, not using tempest code. > > The vision is that you'd have: > > heat/tests/functional/.... > > And tox -e functional would run them. It would require some config for > end points. But the point being it would be fully owned by the project > team. That it could do both blackbox/whitebox testing (and because it's > in the project tree would know things like the data model and could poke > behind the scenes). > > The tight coupling of everything is part of what's gotten us into these > deadlocks, decoupling here is really required in order to reduce the > fragility of the system. > > Since the tempest scenario orchestration tests use heatclient then hopefully it wouldn't be too much effort to forklift them into heat/tests/functional without any tempest dependencies.
We can leave the orchestration api tests where they are until the tempest-lib process results in something ready to use. _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
