On Fri, 27 Jan 2023 at 12:39, Kevin Wolf <kw...@redhat.com> wrote: > > Am 26.01.2023 um 15:28 hat Peter Maydell geschrieben: > > On Thu, 26 Jan 2023 at 14:25, Stefan Hajnoczi <stefa...@gmail.com> wrote: > > > > > > Are you batching pull requests? I used that approach last release > > > cycle. CI takes so long to run that I didn't want to run it for every > > > pull request. Batching worked well overall. > > > > No, I just do one test per pullreq. IME the CI is flaky > > enough that I don't really want to batch it up, and it > > isn't so slow that I build up a backlog of unprocessed > > requests. > > But obviously so slow that we've run out of minutes. It would be good if > this didn't happen every month in the future. > > If it worked well enough for Stefan, I think it would be worth trying to > batch some pull requests going forward. What is the downside of it? If > CI fails and flaky tests seem to be at fault, I assume you just re-run > the job, no matter whether it tests a single pull request or two or > three of them?
It means that if something fails it's harder to see whether it was pullreq A or pullreq B. It also means there's a higher cost to "abandon processing the merge and try a different one to see if that one goes through and come back to this one later", which is also something I sometimes do in an attempt to figure out whether a problem is the usual flaky CI or not. -- PMM
