On 4/17/2017 1:55 PM, Ihar Hrachyshka wrote:
But since it's not executed anywhere in tempest gate, even as
non-voting (?), it's effectively dead code that may be long broken
without anyone knowing. Of course there are consumers of the tests
downstream, but for those consumers it's a tough call to start
depending on the tests if they are not sanity checked by tempest
itself. Wouldn't it make sense to have some job in tempest gate that
would execute those tests (maybe just them to speed up such a job?
maybe non-voting? maybe even as periodic? but there should be
something that keeps it green in long run).
I enabled the job that runs the slow scenario tests in the nova
experimental queue:
https://review.openstack.org/#/c/458676/
Because we were changing some of the encrypted volume code in nova and
that's only exercised by a scenario test that was marked slow, but as
you said I need to run tests to have coverage on those changes. That
works for me in this case, the problem is I had to (1) know those tests
existed and (2) go out of my way looking for their results in a job run
and find out they weren't there, then hunt them out. That's exceptional
for most patches so yes, things are going to slip through and break most
likely because we're not gating on them. I understand why the QA team
did what they did though so I'm not pushing back on that.
--
Thanks,
Matt
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
