Re: Clarifying CI release criteria

2022-01-05 Thread Joshua McKenzie
Could you note that in the wiki in the comments section Ekaterina? https://cwiki.apache.org/confluence/pages/resumedraft.action?draftId=199530280&draftShareId=7c72c252-918c-456b-9859-7d12e8fa9309&; Thanks! On Tue, Jan 4, 2022 at 4:21 PM Ekaterina Dimitrova wrote: > Hey Josh, > Thank you for le

Re: Clarifying CI release criteria

2022-01-04 Thread Ekaterina Dimitrova
Hey Josh, Thank you for leading these discussions and organizing the wiki pages (also from the other mail). I just wanted to mention about point 4 of Pending work - I have a draft version for CircleCI usage, also Andres has updated the rst documents around running tests in a loop, etc. BUT those ar

Re: Clarifying CI release criteria

2022-01-04 Thread Joshua McKenzie
Here's a link to a draft article for the confluence wiki covering what we discussed on this thread: https://cwiki.apache.org/confluence/pages/resumedraft.action?draftId=199530280&draftShareId=7c72c252-918c-456b-9859-7d12e8fa9309&; Assuming this article accurately captures what we discussed here as

Re: Clarifying CI release criteria

2021-12-17 Thread Joshua McKenzie
I'll get this into a draft article on the wiki so we can collab on those 3 outstanding TBD's without further cluttering up the dev list. :) On Fri, Dec 17, 2021 at 11:38 AM Ekaterina Dimitrova wrote: > It’s indeed good call but I thought this will be addressed in a separate > document where we d

Re: Clarifying CI release criteria

2021-12-17 Thread Ekaterina Dimitrova
It’s indeed good call but I thought this will be addressed in a separate document where we discuss required test suites to be run pre-commit. If not - then I guess we should add those things here too? On Fri, 17 Dec 2021 at 11:36, Joshua McKenzie wrote: > Good call; thanks for the reminder. > >

Re: Clarifying CI release criteria

2021-12-17 Thread Joshua McKenzie
Good call; thanks for the reminder. So maybe add a 3.a: Run all new or modified tests through either local or remote multiplexer N (TBD - 50?) times (w/link to instructions, etc) 3.b Non-regressing is defined here... 3.c After merging tickets... On Fri, Dec 17, 2021 at 11:29 AM Brandon Williams

Re: Clarifying CI release criteria

2021-12-17 Thread Brandon Williams
Could we also add something about running new tests through the multiplexer? On Fri, Dec 17, 2021 at 10:23 AM Joshua McKenzie wrote: > > So to clarify it all in one place, the proposed new CI process we should > test for consensus is: > > 1. Canonical CI for a release is ci-cassandra. We can opti

Re: Clarifying CI release criteria

2021-12-17 Thread Joshua McKenzie
So to clarify it all in one place, the proposed new CI process we should test for consensus is: 1. Canonical CI for a release is ci-cassandra. We can optionally, and in practice will, run circle as well but don't codify blocking on that. 2. (NEW) We don't release unless we get a fully green run. 3

Re: Clarifying CI release criteria

2021-12-17 Thread Ekaterina Dimitrova
+1 (nb) on my end too, I second Mick Thanks for putting this together Josh On Fri, 17 Dec 2021 at 10:48, Mick Semb Wever wrote: > > > > > > 3.c: (NEW) After merging tickets, run ci-cassandra (already do this) and > > get an advisory update on the related JIRA for any new errors on the run > of >

Re: Clarifying CI release criteria

2021-12-17 Thread Mick Semb Wever
> > > 3.c: (NEW) After merging tickets, run ci-cassandra (already do this) and > get an advisory update on the related JIRA for any new errors on the run of > the SHA > > I strongly prefer we amend our process with 3.c. +1 Yup, this is the most important missing piece for me. I also wouldn't

Re: Clarifying CI release criteria

2021-12-17 Thread Joshua McKenzie
What if we tried the following: 1. Canonical CI for a release is ci-cassandra. We can optionally, and in practice will, run circle as well but don't codify blocking on that. 2. (NEW) We don't release unless we get a fully green run. 3. Before any merge, you need either a non-regressing (i.e. no ne

Re: Clarifying CI release criteria

2021-12-16 Thread Mick Semb Wever
> > > > ci-cassandra.a.o needs to be our canonical CI > > it's the only one fully usable by a volunteer based > > > only green in both counts as green > > I think today might just be my day to annoy you Mick. :D Sorry! > On the day I'm laid up in bed with a cold. Go for gold :-) I think this is

Re: Clarifying CI release criteria

2021-12-16 Thread Joshua McKenzie
> > ci-cassandra.a.o needs to be our canonical CI it's the only one fully usable by a volunteer based only green in both counts as green I think today might just be my day to annoy you Mick. :D Sorry! I think this is contradictory. We can't require circle to be green for a release if the free

Re: Clarifying CI release criteria

2021-12-16 Thread Mick Semb Wever
> > Mick, did I miss anything? > Yup, ci-cassandra.a.o needs to be our canonical CI because of the reasons you state, and because it's the only one fully usable by a volunteer based PMC, i.e. community donated hardware, controlled by ASF, and no need for a proprietary licence. I think we should

Re: Clarifying CI release criteria

2021-12-16 Thread Ekaterina Dimitrova
As far as I am aware Jenkins is the official CI running on Apache infra. It preserves historical data and every committer and PMC member has access to run all tests pre-commit. We have project builds after every commit. My understanding about CircleCI - free tier is available to everyone but not a

Re: Clarifying CI release criteria

2021-12-15 Thread Berenguer Blasi
Iirc we settled on jenkins being the official CI. Hence we need a green jenkins. I can't find the reference now. Green being all test pass except the 2 or 3 flakies that already defeated 3 committers like the connections test. Last time I looked 4.0 was almost there pending solving the early proce

Clarifying CI release criteria

2021-12-15 Thread Joshua McKenzie
What CI criteria are we using to gate minor and major releases? For example: All tests from all suites on all configurations? A subset? All the test suite columns on the following page? https://ci-cassandra.apache.org/job/Cassandra-trunk/ All the test suites in all combinations on all JDKs on cir