On 7/21/14, 3:38 AM, Matthew Booth wrote:
[snip]
I would like to make the radical proposal that we stop gating on CI
failures. We will continue to run them on every change, but only after
the change has been successfully merged.
Benefits:
* Without rechecks, the gate will use 8 times fewer resources.
* Log analysis is still available to indicate the emergence of races.
* Fixes can be merged quicker.
* Vastly less developer time spent monitoring gate failures.
Costs:
* A rare class of merge bug will make it into master.
Note that the benefits above will also offset the cost of resolving this
rare class of merge bug.
I think this is definitely a move in the right direction, but I'd like
to propose a slight modification: let's cease blocking changes on
*known* CI failures.
More precisely, if Elastic Recheck knows about all the failures that
happened on a test run, treat that test run as successful.
I think this will gain virtually all the benefits you name while still
retaining most of the gate's ability to keep breaking changes out.
As a bonus, it'll encourage people to make Elastic Recheck better.
Currently, the easy path is to just type "recheck no bug" and click
"submit"; it takes a lot less time than scrutinizing log files to guess
at what went wrong. If failures identified by E-R don't block
developers' changes, then the easy path is to improve E-R's checks,
which benefits everyone.
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
