Thanks a lot Jeremy and Andreas for the reference and the added context! On 8/3/17, 10:49 AM, "Jeremy Stanley" <fu...@yuggoth.org> wrote:
>On 2017-08-03 08:15:36 +0200 (+0200), Andreas Jaeger wrote: >[...] >> "A patchset has to be approved to run tests in the gate pipeline. If the >> patchset has failed in the gate pipeline (it will have been approved to >> get into the gate pipeline) a recheck will first run the check jobs and >> if those pass, it will again run the gate jobs. There is no way to only >> run the gate jobs, the check jobs will first be run again." > >The reasons being: > >1. There's no good way to decide how long is too long to wait >between passing jobs in check and running jobs in the gate. We used >to not enforce this "clean check" policy and developers would >repeatedly reverify broken changes back into the gate pipeline over >and over creating a significant amount of additional disruption >because their change had passed check jobs once (perhaps many months >earlier). Now they at least only get to disrupt the gate once when >that change gets approved, but after that it won't be able to make >it back into the gate until a fixed revision is uploaded and so >doesn't further slow down the merging of unrelated changes. > >2. If a change passes jobs once (in check) and then fails later (in >the gate) then there's a fair chance it's introducing a >nondeterministic bug (one which only manifests sometimes but not on >every run). Back when we used to allow reverification directly in >the gate pipeline for changes which passed check, we had people >rechecking flaky changes until they passed and then reverifying them >over and over after approval until they made them through the gate. >Under these conditions a recheck followed by a reverify could merge >changes which failed jobs 50% of the time; 9 rechecks and 9 >reverifies could merge a change which failed jobs 90% of the time on >average. With the current requirements to pass both check and gate >in series, it takes on average 3 rechecks to merge a 50% failing >change and 99 rechecks to merge a 90% failing change. > >So basically if a change fails in the gate pipeline, there's good >reason for it to get increased scrutiny at least in the form of >trying the jobs again in the check pipeline before going back to the >gate once more. >-- >Jeremy Stanley >__________________________________________________________________________ >OpenStack Development Mailing List (not for usage questions) >Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
