One more thing: 3. If test flakiness is introduced by a recent PR, it's appropriate to revert said PR vs disabling the flaky tests.
Ismael On Sat, Nov 11, 2023, 8:45 AM Ismael Juma <m...@ismaeljuma.com> wrote: > Hi David, > > I would be fine with: > 1. Only allowing merges if the build is green > 2. Disabling all flaky tests that aren't fixed within a week. That is, if > a test is flaky for more than a week, it should be automatically disabled > (it doesn't add any value since it gets ignored). > > We need both to make this work, if you just do step 1, then we will be > stuck with no ability to merge anything. > > Ismael > > > > On Sat, Nov 11, 2023, 2:02 AM David Jacot <dja...@confluent.io.invalid> > wrote: > >> Hi all, >> >> The state of our CI worries me a lot. Just this week, we merged two PRs >> with compilation errors and one PR introducing persistent failures. This >> really hurts the quality and the velocity of the project and it basically >> defeats the purpose of having a CI because we tend to ignore it nowadays. >> >> Should we continue to merge without a green build? No! We should not so I >> propose to prevent merging a pull request without a green build. This is a >> really simple and bold move that will prevent us from introducing >> regressions and will improve the overall health of the project. At the >> same >> time, I think that we should disable all the known flaky tests, raise >> jiras >> for them, find an owner for each of them, and fix them. >> >> What do you think? >> >> Best, >> David >> >
