Could you note that in the wiki in the comments section Ekaterina? https://cwiki.apache.org/confluence/pages/resumedraft.action?draftId=199530280&draftShareId=7c72c252-918c-456b-9859-7d12e8fa9309&;
Thanks! On Tue, Jan 4, 2022 at 4:21 PM Ekaterina Dimitrova <e.dimitr...@gmail.com> wrote: > Hey Josh, > Thank you for leading these discussions and organizing the wiki pages > (also from the other mail). > I just wanted to mention about point 4 of Pending work - I have a draft > version for CircleCI usage, also Andres has updated the rst documents > around running tests in a loop, etc. > BUT those are pending on the merge of the ascii docs (to convert his work) > and add the correct links and any other changes that have happened since I > wrote it last year. Just saying for awareness before anyone decides to make > a write up on that not knowing a draft already exists. > > Thanks, > Ekaterina > > On Tue, 4 Jan 2022 at 14:51, Joshua McKenzie <jmcken...@apache.org> wrote: > >> Here's a link to a draft article for the confluence wiki covering what we >> discussed on this thread: >> https://cwiki.apache.org/confluence/pages/resumedraft.action?draftId=199530280&draftShareId=7c72c252-918c-456b-9859-7d12e8fa9309&; >> >> Assuming this article accurately captures what we discussed here as well >> as outstanding work, I'll get it published and integrated with the >> confluence wiki and get the remainder of the work into a JIRA epic to be >> tracked. >> >> On Fri, Dec 17, 2021 at 4:41 PM Joshua McKenzie <jmcken...@apache.org> >> wrote: >> >>> I'll get this into a draft article on the wiki so we can collab on those >>> 3 outstanding TBD's without further cluttering up the dev list. :) >>> >>> On Fri, Dec 17, 2021 at 11:38 AM Ekaterina Dimitrova < >>> e.dimitr...@gmail.com> wrote: >>> >>>> It’s indeed good call but I thought this will be addressed in a separate >>>> document where we discuss required test suites to be run pre-commit. If >>>> not >>>> - then I guess we should add those things here too? >>>> >>>> On Fri, 17 Dec 2021 at 11:36, Joshua McKenzie <jmcken...@apache.org> >>>> wrote: >>>> >>>> > Good call; thanks for the reminder. >>>> > >>>> > So maybe add a >>>> > >>>> > 3.a: Run all new or modified tests through either local or remote >>>> > multiplexer N (TBD - 50?) times (w/link to instructions, etc) >>>> > 3.b Non-regressing is defined here... >>>> > 3.c After merging tickets... >>>> > >>>> > On Fri, Dec 17, 2021 at 11:29 AM Brandon Williams <dri...@gmail.com> >>>> > wrote: >>>> > >>>> > > Could we also add something about running new tests through the >>>> > > multiplexer? >>>> > > >>>> > > On Fri, Dec 17, 2021 at 10:23 AM Joshua McKenzie < >>>> jmcken...@apache.org> >>>> > > wrote: >>>> > > > >>>> > > > So to clarify it all in one place, the proposed new CI process we >>>> > should >>>> > > > test for consensus is: >>>> > > > >>>> > > > 1. Canonical CI for a release is ci-cassandra. We can optionally, >>>> and >>>> > in >>>> > > > practice will, run circle as well but don't codify blocking on >>>> that. >>>> > > > 2. (NEW) We don't release unless we get a fully green run. >>>> > > > 3. Before any merge, you need either a non-regressing (i.e. no new >>>> > > > failures) run of circleci with a (specific suite of tests TBD) or >>>> of >>>> > > > ci-cassandra. >>>> > > > 3.a Non-regressing is defined here as "Doesn't introduce any >>>> new >>>> > > test >>>> > > > failures; any new failures in CI are clearly not attributable to >>>> this >>>> > > diff" >>>> > > > 3.b: (NEW) After merging tickets, ci-cassandra runs against >>>> the >>>> > SHA >>>> > > > and the author gets an advisory update on the related JIRA for >>>> any new >>>> > > > errors on CI. The author of the ticket will take point on >>>> triaging this >>>> > > new >>>> > > > failure and either fixing (if clearly reproducible or related to >>>> their >>>> > > > work) or opening a JIRA for the intermittent failure and linking >>>> it in >>>> > > > butler (https://butler.cassandra.apache.org/#/) >>>> > > > 4. (NEW) The Build Lead role + Butler catches and documents all >>>> > failures >>>> > > > and anything that slips through the procedural cracks in 3.b; >>>> > resourcing >>>> > > > for fixing flakey tests TBD >>>> > > > >>>> > > > Our two TBD we can tackle separately from consensus on the above: >>>> > > > 1. Suite of tests on circle required to be considered ready for >>>> merge >>>> > > > 2. How we resource fixing flakey tests that are functionally >>>> impossible >>>> > > to >>>> > > > attribute without essentially fixing the flake >>>> > > > >>>> > > > On Fri, Dec 17, 2021 at 10:56 AM Ekaterina Dimitrova < >>>> > > e.dimitr...@gmail.com> >>>> > > > wrote: >>>> > > > >>>> > > > > +1 (nb) on my end too, I second Mick >>>> > > > > Thanks for putting this together Josh >>>> > > > > >>>> > > > > On Fri, 17 Dec 2021 at 10:48, Mick Semb Wever <m...@apache.org> >>>> > wrote: >>>> > > > > >>>> > > > > > > >>>> > > > > > > >>>> > > > > > > 3.c: (NEW) After merging tickets, run ci-cassandra (already >>>> do >>>> > > this) >>>> > > > > and >>>> > > > > > > get an advisory update on the related JIRA for any new >>>> errors on >>>> > > the >>>> > > > > run >>>> > > > > > of >>>> > > > > > > the SHA >>>> > > > > > > >>>> > > > > > > I strongly prefer we amend our process with 3.c. >>>> > > > > > >>>> > > > > > >>>> > > > > > >>>> > > > > > +1 Yup, this is the most important missing piece for me. >>>> > > > > > >>>> > > > > > I also wouldn't mind we word the responsibility of the author >>>> at >>>> > > > > > post-commit fault to be involved/leading in the fix. This >>>> > > incentivises >>>> > > > > > people to do 2+3 properly, and not push it onto the build >>>> role. >>>> > > > > > >>>> > > > > >>>> > > >>>> > > >>>> --------------------------------------------------------------------- >>>> > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org >>>> > > For additional commands, e-mail: dev-h...@cassandra.apache.org >>>> > > >>>> > > >>>> > >>>> >>>
