On Feb 12, 2014, at 1:59 PM, Sean Dague <s...@dague.net> wrote: > On 02/12/2014 04:25 PM, Maru Newby wrote: >> >> On Feb 12, 2014, at 12:36 PM, Sean Dague <s...@dague.net> wrote: >> >>> On 02/12/2014 01:48 PM, Maru Newby wrote: >>>> At the last 2 summits, I've suggested that API tests could be maintained >>>> in the Neutron tree and reused by Tempest. I've finally submitted some >>>> patches that demonstrate this concept: >>>> >>>> https://review.openstack.org/#/c/72585/ (implements a unit test for the >>>> lifecycle of the network resource) >>>> https://review.openstack.org/#/c/72588/ (runs the test with tempest rest >>>> clients) >>>> >>>> My hope is to make API test maintenance a responsibility of the Neutron >>>> team. The API compatibility of each Neutron plugin has to be validated by >>>> Neutron tests anyway, and if the tests are structured as I am proposing, >>>> Tempest can reuse those efforts rather than duplicating them. >>>> >>>> I've added this topic to this week's agenda, and I would really appreciate >>>> it interested parties would take a look at the patches in question to >>>> prepare themselves to participate in the discussion. >>>> >>>> >>>> m. >>>> _______________________________________________ >>>> OpenStack-dev mailing list >>>> OpenStack-dev@lists.openstack.org >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>> >>> Realistically, having API tests duplicated in the Tempest tree is a >>> feature, not a bug. >>> >>> tempest/api is there for double book keep accounting, and it has been >>> really effective at preventing accidental breakage of our APIs (which >>> used to happen all the time), so I don't think putting API testing in >>> neutron obviates that. >> >> Given how limited our testing resources are, might it be worth considering >> whether 'double-entry accounting' is actually the best way to preventing >> accidental breakage going forward? Might reasonable alternatives exist, >> such as clearly separating api tests from other tests in the neutron tree >> and giving review oversight only to qualified individuals? > > Our direct experience is that if we don't do this, within 2 weeks some > project will have landed API breaking changes. This approach actually > takes a lot of review load off the core reviewers, so reverting to a > model which puts more work back on the review team (given the current > review load), isn't something I think we want.
Just so I'm clear, is there anything I could say that would change your mind? m. > > I get that there is a cost with this. But there is a cost of all of > this. And because API tests should be write once for the API (and > specifically not evolving), the upfront cost vs. the continually paid > cost of review time in tree has been the better trade off. > > -Sean > > -- > Sean Dague > Samsung Research America > s...@dague.net / sean.da...@samsung.com > http://dague.net > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
