On 01/12/15 06:22, Steven Hardy wrote:
> +1
>
> I don't think it hurts to turn it on, but tbh I'm uncomfortable with the
> mental overhead of a non-voting job that I have to manually treat as a
> voting job. If it's stable enough to make it a voting job, I'd prefer we
> just make it voting. And if it's not then I'd like to see it be made
> stable enough to be a voting job and then make it voting.
>
> This is roughly where I sit as well -- if it's non-voting, experience
> tells me that it will largely be ignored, and as such, isn't a good use of
> resources.
I'm sure you can appreciate it's something of a chicken/egg problem though
- if everyone always ignores non-voting jobs, they never become voting.
That effect is magnified with TripleO though, because it consumes so many
OpenStack projects, any one of which has the capability to break our CI, so
in an ideal world we'd have voting feedback on all-the-things, but that's
not where we are right now due in large-part to the steady stream of
regressions (from Heat, Ironic and other projects).
Yeah, but when it does break the problem becomes that reviewers now have
to debug TripleO tests on every single patchset to see if they broke it
(or made it worse) or not. Pretty soon you never know whether you should
be ignoring it or not, and... we all know how this story ends.
Maybe it would help if there was a periodic job running TripleO tests
against master and posting the results somewhere. If I could go to
aretripleocheckfailuresreal.com and it just says YES or NO in giant
8-inch high letters, that would help a lot. The test could run let's say
once an hour and if the last 3 all passed then we'd call that YES.
Even better would be if we could tie this into the gate, so that it runs
as a gating test when the tests are working and not when they aren't. I
don't think infra will thank me for suggesting that though ;)
It also occurs to me that the trade-off may be different for stable
branches. It might well be possible to start gating stable/liberty Heat
against TripleO right away.
cheers,
Zane.
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
