We might want to consider pushing out the branch point a week anyway to help people get CTS in order?
On Fri, Jan 7, 2022 at 1:08 PM Ian Romanick <i...@freedesktop.org> wrote: > Blarg. That all sounds awful. I think (hope!) I speak for everyone when > I say that we all appreciate your and daniels' efforts to keep this big > piece of machinery working. > > If the problems persist much longer, should we consider pushing out the > 22.0 branch point? > > On 1/6/22 9:36 PM, Emma Anholt wrote: > > As you've probably noticed, there have been issues with git access > > this week. The fd.o sysadmins are desperately trying to stay on > > vacation because they do deserve a break, but have still been working > > on the problem and a couple of solutions haven't worked out yet. > > Hopefully we'll have some news soon. > > > > Due to these ongoing git timeouts, our CI runners have been getting > > bogged down with stalled jobs and causing a lot of spurious failures > > where the pipeline doesn't get all its jobs assigned to runners before > > Marge gives up. Today, I asked daniels to bump Marge's pipeline > > timeout to 4 hours (up from 1). To get MRs flowing at a similar rate > > despite the longer total pipeline times, we also enabled batch mode as > > described at > https://github.com/smarkets/marge-bot/blob/master/README.md#batching-merge-requests > . > > > > It means there are now theoretical cases as described in the README > > where Marge might merge a set of code that leaves main broken. > > However, those cases are pretty obscure, and I expect that failure > > rate to be much lower than the existing "you can merge flaky code" > > failure rate and worth the risk. > > > > Hopefully this gets us all productive again. >
