On 2013-08-10 14:20:40 +0000 (+0000), Bob Ball wrote: [...] > My understanding at the moment is that there is a check > immediately prior to merging to make sure that there aren't any > -2's added by core reviewers between the merge being approved and > the gating jobs completing - is that right? If so, I'm not sure I > understand the differences between a reviewer preventing a merge > in this fashion and a known test failure from preventing it. [...]
Just so I understand the suggestion, you're saying you can make smokestack test the changes stacked in the same sequence they're being started by zuul, watch zuul's status mirroring the various change ejections/test restarts... and finish testing on a given set of scenarios before zuul does? (Remember that zuul can effectively start testing an arbitrary number of sequenced changes in parallel so any resource congestion at the smokestack end could quickly cause it to fall behind at load.) You had mentioned something about smokestack adding a -2 when it begins testing and removing it on success, but if it doesn't complete before zuul is ready to merge those commits then how do we differentiate between smokestack still running (so we know to pause the queue) and smokestack leaving that -2 indefinitely because the test failed? Also worth noting, gerrit doesn't clear a -2 on a new patchset upload like it does with a -1 or a positive vote, so the proposed solution would additionally need to take care of rescinding those -2 votes itself when new patchsets arrive to correct the failure. This all sounds very fragile and a probably a non-trivial amount of effort to adapt to the situation you describe. I'm in agreement with the other voiced opinions that we should continue to work toward testing everything we can in a well-integrated and timely manner, provide a way for less tightly-coupled systems to chime in with automated advisory notes which can help reviewers, and always be prepared deal with the unfortunate but inevitable clean-up when something merges which breaks a particular configuration (we have to do that anyway because nobody will every be able to write tests which exercise every possible eventuality). -- Jeremy Stanley _______________________________________________ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
