On Wed, Mar 23, 2022 at 12:44:09PM +1300, Thomas Munro wrote: > On Mon, Mar 21, 2022 at 12:46 PM Thomas Munro <thomas.mu...@gmail.com> wrote: > > On Mon, Mar 21, 2022 at 12:23 PM Thomas Munro <thomas.mu...@gmail.com> > > wrote: > > > On Mon, Mar 21, 2022 at 1:58 AM Matthias van de Meent > > > <boekewurm+postg...@gmail.com> wrote: > > > > Additionally, are there plans to validate commits of the main branch > > > > before using them as a base for CF entries, so that "bad" commits on > > > > master won't impact CFbot results as easy? > > > > > > How do you see this working? > > > > [Now with more coffee on board] Oh, right, I see, you're probably > > thinking that we could look at > > https://github.com/postgres/postgres/commits/master and take the most > > recent passing commit as a base. Hmm, interesting idea. > > A nice case in point today: everything is breaking on Windows due to a > commit in master, which could easily be avoided by looking back a > certain distance for a passing commit from postgres/postgres to use as > a base. Let's me see if this is easy to fix... > > https://www.postgresql.org/message-id/20220322231311.GK28503%40telsasoft.com
I suggest not to make it too sophisticated. If something is broken, the CI should show that rather than presenting a misleading conclusion. Maybe you could keep track of how many consecutive, *new* failures there've been (which were passing on the previous run for that task, for that patch) and delay if it's more than (say) 5. For bonus points, queue a rerun of all the failed tasks once something passes. If you create a page to show the queue, maybe it should show the history of results, too. And maybe there should be a history of results for each patch. If you implement interactive buttons, maybe it could allow re-queueing some recent failures (add to end of queue). -- Justin
