As far as I am aware Jenkins is the official CI running on Apache infra. It preserves historical data and every committer and PMC member has access to run all tests pre-commit. We have project builds after every commit.
My understanding about CircleCI - free tier is available to everyone but not all tests can be executed with the free account because of lack of enough resources. Not everyone is willing to pay for premium accounts. We don’t have project account which runs the suite post-commit and has project build history. Pre-4.0 we used Jenkins to confirm CI situation. Also, in my opinion actually CircleCI sometimes reveals more issues than Jenkins as people run it with different configurations and more things can pop up. In regards to the Build Lead role, Butler gets the info from Jenkins and the history is available to everyone - contributor, committer, PMC. “ Also, if a bunch of us are using Circle > pre-commit and we're seeing test flakes unique to the ASF Jenkins execution > environment, we're going to naturally accumulate debt as we lead up to a > release if we block on ASF Jenkins being green.” If we get into the cadence of checking regularly post-commit in Jenkins, I hope this won’t happen. Mick, did I miss anything? On Thu, 16 Dec 2021 at 2:28, Berenguer Blasi <berenguerbl...@gmail.com> wrote: > Iirc we settled on jenkins being the official CI. Hence we need a green > jenkins. I can't find the reference now. > > Green being all test pass except the 2 or 3 flakies that already > defeated 3 committers like the connections test. Last time I looked 4.0 > was almost there pending solving the early process termination failures. > > On 15/12/21 23:44, Joshua McKenzie wrote: > > What CI criteria are we using to gate minor and major releases? For > > example: All tests from all suites on all configurations? A subset? > > > > All the test suite columns on the following page? > > https://ci-cassandra.apache.org/job/Cassandra-trunk/ > > > > All the test suites in all combinations on all JDKs on circle-ci? > > > > This is particularly relevant to how we craft the Build Lead role > w/regards > > to what we'll be triaging. Also, if a bunch of us are using Circle > > pre-commit and we're seeing test flakes unique to the ASF Jenkins > execution > > environment, we're going to naturally accumulate debt as we lead up to a > > release if we block on ASF Jenkins being green. > > > > Notably, I can't find this clarified in our documentation on testing or > the > > release process, nor is it clarified on the confluence wiki article on > the > > Release Lifecycle. > > > > Thanks. > > > > ~Josh > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
