I think the goal is to say "how could we get some working version of TCM/Accord into people's hands to try out at/by Summit?" That's all. People are eager to see it and try it out.
> On Oct 31, 2023, at 12:16 PM, Benedict <bened...@apache.org> wrote: > > No, if I understand it correctly we’re in weird hypothetical land where > people are inventing new release types (“preview”) to avoid merging TCM[1] in > the event we want to cut a 5.1 release from the PR prior to the summit if > there’s some handful of failing tests in the PR. > > This may or may not be a waste of everyone’s time. > > Jeremiah, I’m not questioning: it was procedurally invalid. How we handle > that is, as always, a matter for community decision making. > > [1] how this helps isn’t entirely clear > >> On 31 Oct 2023, at 17:08, Paulo Motta <pauloricard...@gmail.com> wrote: >> >> >> Even if it was not formally prescribed as far as I understand, we have been >> following the "only merge on Green CI" custom as much as possible for the >> past several years. Is the proposal to relax this rule for 5.0? >> >> On Tue, Oct 31, 2023 at 1:02 PM Jeremiah Jordan <jeremiah.jor...@gmail.com >> <mailto:jeremiah.jor...@gmail.com>> wrote: >>> You are free to argue validity. I am just stating what I see on the >>> mailing list and in the wiki. We had a vote which was called passing and >>> was not contested at that time. The vote was on a process which includes >>> as #3 in the list: >>> >>> Before a merge, a committer needs either a non-regressing (i.e. no new >>> failures) run of circleci with the required test suites (TBD; see below) or >>> of ci-cassandra. >>> Non-regressing is defined here as "Doesn't introduce any new test failures; >>> any new failures in CI are clearly not attributable to this diff" >>> (NEW) After merging tickets, ci-cassandra runs against the SHA and the >>> author gets an advisory update on the related JIRA for any new errors on >>> CI. The author of the ticket will take point on triaging this new failure >>> and either fixing (if clearly reproducible or related to their work) or >>> opening a JIRA for the intermittent failure and linking it in butler >>> (https://butler.cassandra.apache.org/#/) >>> >>> Which clearly says that before merge we ensure there are no known new >>> regressions to CI. >>> >>> The allowance for releases without CI being green, and merges without the >>> CI being completely green are from the fact that our trunk CI has rarely >>> been completely green, so we allow merging things which do not introduce >>> NEW regressions, and we allow releases with known regressions that are >>> deemed acceptable. >>> >>> We can indeed always vote to override it, and if it comes to that we can >>> consider that as an option. >>> >>> -Jeremiah >>> >>> On Oct 31, 2023 at 11:41:29 AM, Benedict <bened...@apache.org >>> <mailto:bened...@apache.org>> wrote: >>>> That vote thread also did not reach the threshold; it was incorrectly >>>> counted, as committer votes are not binding for procedural changes. I >>>> counted at most 8 PMC +1 votes. >>>> >>>> The focus of that thread was also clearly GA releases and merges on such >>>> branches, since there was a focus on releases being failure-free. But this >>>> predates the more general release lifecycle vote that allows for alphas to >>>> have failing tests - which logically would be impossible if nothing were >>>> merged with failing or flaky tests. >>>> >>>> Either way, the vote and discussion specifically allow for this to be >>>> overridden. >>>> >>>> 🤷♀️ >>>> >>>>> On 31 Oct 2023, at 16:29, Jeremiah Jordan <jeremiah.jor...@gmail.com >>>>> <mailto:jeremiah.jor...@gmail.com>> wrote: >>>>> >>>>> >>>>> I never said there was a need for green CI for alpha. We do have a >>>>> requirement for not merging things to trunk that have known regressions >>>>> in CI. >>>>> Vote here: >>>>> https://lists.apache.org/thread/j34mrgcy9wrtn04nwwymgm6893h0xwo9 >>>>> >>>>> >>>>> >>>>> On Oct 31, 2023 at 3:23:48 AM, Benedict <bened...@apache.org >>>>> <mailto:bened...@apache.org>> wrote: >>>>>> There is no requirement for green CI on alpha. We voted last year to >>>>>> require running all tests before commit and to require green CI for beta >>>>>> releases. This vote was invalid because it didn’t reach the vote floor >>>>>> for a procedural change but anyway is not inconsistent with knowingly >>>>>> and selectively merging work without green CI. >>>>>> >>>>>> If we reach the summit we should take a look at the state of the PRs and >>>>>> make a decision about if they are alpha quality; if so, and we want a >>>>>> release, we should simply merge it and release. Making up a new release >>>>>> type when the work meets alpha standard to avoid an arbitrary and not >>>>>> mandated commit bar seems the definition of silly. >>>>>> >>>>>>> On 31 Oct 2023, at 04:34, J. D. Jordan <jeremiah.jor...@gmail.com >>>>>>> <mailto:jeremiah.jor...@gmail.com>> wrote: >>>>>>> >>>>>>> >>>>>>> That is my understanding as well. If the TCM and Accord based on TCM >>>>>>> branches are ready to commit by ~12/1 we can cut a 5.1 branch and then >>>>>>> a 5.1-alpha release. >>>>>>> Where “ready to commit” means our usual things of two committer +1 and >>>>>>> green CI etc. >>>>>>> >>>>>>> If we are not ready to commit then I propose that as long as everything >>>>>>> in the accord+tcm Apache repo branch has had two committer +1’s, but >>>>>>> maybe people are still working on fixes for getting CI green or >>>>>>> similar, we cut a 5.1-preview build from the feature branch to vote on >>>>>>> with known issues documented. This would not be the preferred path, >>>>>>> but would be a way to have a voted on release for summit. >>>>>>> >>>>>>> -Jeremiah >>>>>>> >>>>>>>> On Oct 30, 2023, at 5:59 PM, Mick Semb Wever <m...@apache.org >>>>>>>> <mailto:m...@apache.org>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Hoping we can get clarity on this. >>>>>>>> >>>>>>>> The proposal was, once TCM and Accord merges to trunk, then >>>>>>>> immediately branch cassandra-5.1 and cut an immediate 5.1-alpha1 >>>>>>>> release. >>>>>>>> >>>>>>>> This was to focus on stabilising TCM and Accord as soon as it lands, >>>>>>>> hence the immediate branching. >>>>>>>> >>>>>>>> And the alpha release as that is what our Release Lifecycle states it >>>>>>>> to be. >>>>>>>> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle >>>>>>>> >>>>>>>> >>>>>>>> My understanding is that there was no squeezing in extra features into >>>>>>>> 5.1 after TCM+Accord lands, and there's no need for a "preview" >>>>>>>> release – we move straight to the alpha, as our lifecycle states. And >>>>>>>> we will describe all usability shortcomings and bugs with the alpha, >>>>>>>> our lifecycle docs permit this, if we feel the need to. >>>>>>>> >>>>>>>> All this said, if TCM does not merge before the Summit, and we want to >>>>>>>> get a release into user hands, it has been suggested we cut a preview >>>>>>>> release 5.1-preview1 off the feature branch. This is a different >>>>>>>> scenario, and only a mitigation plan. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Thu, 26 Oct 2023 at 14:20, Benedict <bened...@apache.org >>>>>>>> <mailto:bened...@apache.org>> wrote: >>>>>>>>> The time to stabilise is orthogonal to the time we branch. Once we >>>>>>>>> branch we stop accepting new features for the branch, and work to >>>>>>>>> stabilise. >>>>>>>>> >>>>>>>>> My understanding is we will branch as soon as we have a viable alpha >>>>>>>>> containing TCM and Accord. That means pretty soon after they land in >>>>>>>>> the project, which we expect to be around the summit. >>>>>>>>> >>>>>>>>> If this isn’t the expectation we should make that clear, as it will >>>>>>>>> affect how this decision is made. >>>>>>>>> >>>>>>>>>> On 26 Oct 2023, at 10:14, Benjamin Lerer <b.le...@gmail.com >>>>>>>>>> <mailto:b.le...@gmail.com>> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Regarding the release of 5.1, I understood the proposal to be that >>>>>>>>>>> we cut an actual alpha, thereby sealing the 5.1 release from new >>>>>>>>>>> features. Only features merged before we cut the alpha would be >>>>>>>>>>> permitted, and the alpha should be cut as soon as practicable. What >>>>>>>>>>> exactly would we be waiting for? >>>>>>>>>> >>>>>>>>>> The problem I believe is about expectations. It seems that your >>>>>>>>>> expectation is that a release with only TCM and Accord will reach GA >>>>>>>>>> quickly. Based on the time it took us to release 4.1, I am simply >>>>>>>>>> expecting more delays (a GA around end of May, June). In which case >>>>>>>>>> it seems to me that we could be interested in shipping more stuff in >>>>>>>>>> the meantime (thinking of CASSANDRA-15254 or CEP-29 for example). >>>>>>>>>> I do not have a strong opinion, I just want to make sure that we all >>>>>>>>>> share the same understanding and fully understand what we agree >>>>>>>>>> upon. >>>>>>>>>> >>>>>>>>>> Le jeu. 26 oct. 2023 à 10:59, Benjamin Lerer <b.le...@gmail.com >>>>>>>>>> <mailto:b.le...@gmail.com>> a écrit : >>>>>>>>>>>> I am surprised this needs to be said, but - especially for >>>>>>>>>>>> long-running CEPs - you must involve yourself early, and certainly >>>>>>>>>>>> within some reasonable time of being notified the work is ready >>>>>>>>>>>> for broader input and review. In this case, more than six months >>>>>>>>>>>> ago. >>>>>>>>>>> >>>>>>>>>>> It is unfortunately more complicated than that because six month >>>>>>>>>>> ago Ekaterina and I were working on supporting Java 17 and dropping >>>>>>>>>>> Java 8 which was needed by different ongoing works. We both missed >>>>>>>>>>> the announcement that TCM was ready for review and anyway would not >>>>>>>>>>> have been available at that time. Maxim has asked me ages ago for a >>>>>>>>>>> review of CASSANDRA-15254 >>>>>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-15254> more than >>>>>>>>>>> 6 months ago and I have not been able to help him so far. We all >>>>>>>>>>> have a limited bandwidth and can miss some announcements. >>>>>>>>>>> >>>>>>>>>>> The project has grown and a lot of things are going on in parallel. >>>>>>>>>>> There are also more interdependencies between the different >>>>>>>>>>> projects. In my opinion what we are lacking is a global overview of >>>>>>>>>>> the different things going on in the project and some rough ideas >>>>>>>>>>> of the status of the different significant pieces. It would allow >>>>>>>>>>> us to better organize ourselves. >>>>>>>>>>> >>>>>>>>>>> Le jeu. 26 oct. 2023 à 00:26, Benedict <bened...@apache.org >>>>>>>>>>> <mailto:bened...@apache.org>> a écrit : >>>>>>>>>>>> I have spoken privately with Ekaterina, and to clear up some >>>>>>>>>>>> possible ambiguity: I realise nobody has demanded a delay to this >>>>>>>>>>>> work to conduct additional reviews; a couple of folk have however >>>>>>>>>>>> said they would prefer one. >>>>>>>>>>>> >>>>>>>>>>>> My point is that, as a community, we need to work on ensuring folk >>>>>>>>>>>> that care about a CEP participate at an appropriate time. If they >>>>>>>>>>>> aren’t able to, the consequences of that are for them to bear. >>>>>>>>>>>> >>>>>>>>>>>> We should be working to avoid surprises as CEP start to land. To >>>>>>>>>>>> this end, I think we should work on some additional paragraphs for >>>>>>>>>>>> the governance doc covering expectations around the landing of >>>>>>>>>>>> CEPs. >>>>>>>>>>>> >>>>>>>>>>>>> On 25 Oct 2023, at 21:55, Benedict <bened...@apache.org >>>>>>>>>>>>> <mailto:bened...@apache.org>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I am surprised this needs to be said, but - especially for >>>>>>>>>>>>> long-running CEPs - you must involve yourself early, and >>>>>>>>>>>>> certainly within some reasonable time of being notified the work >>>>>>>>>>>>> is ready for broader input and review. In this case, more than >>>>>>>>>>>>> six months ago. >>>>>>>>>>>>> >>>>>>>>>>>>> This isn’t the first time this has happened, and it is >>>>>>>>>>>>> disappointing to see it again. Clearly we need to make this >>>>>>>>>>>>> explicit in the guidance docs. >>>>>>>>>>>>> >>>>>>>>>>>>> Regarding the release of 5.1, I understood the proposal to be >>>>>>>>>>>>> that we cut an actual alpha, thereby sealing the 5.1 release from >>>>>>>>>>>>> new features. Only features merged before we cut the alpha would >>>>>>>>>>>>> be permitted, and the alpha should be cut as soon as practicable. >>>>>>>>>>>>> What exactly would we be waiting for? >>>>>>>>>>>>> >>>>>>>>>>>>> If we don’t have a clear and near-term trigger for branching 5.1 >>>>>>>>>>>>> for its own release, shortly after Accord and TCM merge, then I >>>>>>>>>>>>> am in favour of instead delaying 5.0. >>>>>>>>>>>>> >>>>>>>>>>>>>> On 25 Oct 2023, at 19:40, Mick Semb Wever <m...@apache.org >>>>>>>>>>>>>> <mailto:m...@apache.org>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I'm open to the suggestions of not branching cassandra-5.1 >>>>>>>>>>>>>> and/or naming a preview release something other than 5.1-alpha1. >>>>>>>>>>>>>> >>>>>>>>>>>>>> But… the codebases and release process (and upgrade tests) do >>>>>>>>>>>>>> not currently support releases with qualifiers that are not >>>>>>>>>>>>>> alpha, beta, or rc. We can add a preview qualifier to this >>>>>>>>>>>>>> list, but I would not want to block getting a preview release >>>>>>>>>>>>>> out only because this wasn't yet in place. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hence the proposal used 5.1-alpha1 simply because that's what we >>>>>>>>>>>>>> know we can do today. An alpha release also means (typically) >>>>>>>>>>>>>> the branch. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Is anyone up for looking into adding a "preview" qualifier to >>>>>>>>>>>>>> our release process? >>>>>>>>>>>>>> This may also solve our previous discussions and desire to have >>>>>>>>>>>>>> quarterly releases that folk can use through the trunk dev cycle. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Personally, with my understanding of timelines in front of us to >>>>>>>>>>>>>> fully review and stabilise tcm, it makes sense to branch it as >>>>>>>>>>>>>> soon as it's merged. It's easiest to stabilise it on a branch, >>>>>>>>>>>>>> and there's definitely the desire and demand to do so, so it >>>>>>>>>>>>>> won't be getting forgotten or down-prioritised. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, 25 Oct 2023 at 18:07, Jeremiah Jordan >>>>>>>>>>>>>> <jeremiah.jor...@gmail.com <mailto:jeremiah.jor...@gmail.com>> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> If we do a 5.1 release why not take it as an opportunity to >>>>>>>>>>>>>>>> release more things. I am not saying that we will. Just that >>>>>>>>>>>>>>>> we should let that door open. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Agreed. This is the reason I brought up the possibility of not >>>>>>>>>>>>>>> branching off 5.1 immediately. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Oct 25, 2023 at 3:17:13 AM, Benjamin Lerer >>>>>>>>>>>>>>> <b.le...@gmail.com <mailto:b.le...@gmail.com>> wrote: >>>>>>>>>>>>>>>> The proposal includes 3 things: >>>>>>>>>>>>>>>> 1. Do not include TCM and Accord in 5.0 to avoid delaying 5.0 >>>>>>>>>>>>>>>> 2. The next release will be 5.1 and will include only Accord >>>>>>>>>>>>>>>> and TCM >>>>>>>>>>>>>>>> 3. Merge TCM and Accord right now in 5.1 (making an initial >>>>>>>>>>>>>>>> release) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I am fine with question 1 and do not have a strong opinion on >>>>>>>>>>>>>>>> which way to go. >>>>>>>>>>>>>>>> 2. Means that every new feature will have to wait for post 5.1 >>>>>>>>>>>>>>>> even if it is ready before 5.1 is stabilized and shipped. If >>>>>>>>>>>>>>>> we do a 5.1 release why not take it as an opportunity to >>>>>>>>>>>>>>>> release more things. I am not saying that we will. Just that >>>>>>>>>>>>>>>> we should let that door open. >>>>>>>>>>>>>>>> 3. There is a need to merge TCM and Accord as maintaining >>>>>>>>>>>>>>>> those separate branches is costly in terms of time and energy. >>>>>>>>>>>>>>>> I fully understand that. On the other hand merging TCM and >>>>>>>>>>>>>>>> Accord will make the TCM review harder and I do believe that >>>>>>>>>>>>>>>> this second round of review is valuable as it already >>>>>>>>>>>>>>>> uncovered a valid issue. Nevertheless, I am fine with merging >>>>>>>>>>>>>>>> TCM as soon as it passes CI and continuing the review after >>>>>>>>>>>>>>>> the merge. If we cannot meet at least that quality level >>>>>>>>>>>>>>>> (Green CI) we should not merge just for creating an 5.1.alpha >>>>>>>>>>>>>>>> release for the summit. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Now, I am totally fine with a preview release without >>>>>>>>>>>>>>>> numbering and with big warnings that will only serve as a >>>>>>>>>>>>>>>> preview for the summit. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Le mer. 25 oct. 2023 à 06:33, Berenguer Blasi >>>>>>>>>>>>>>>> <berenguerbl...@gmail.com <mailto:berenguerbl...@gmail.com>> a >>>>>>>>>>>>>>>> écrit : >>>>>>>>>>>>>>>>> I also think there's many good new features in 5.0 already >>>>>>>>>>>>>>>>> they'd make a >>>>>>>>>>>>>>>>> good release even on their own. My 2 cts. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 24/10/23 23:20, Brandon Williams wrote: >>>>>>>>>>>>>>>>> > The catch here is that we don't publish docker images >>>>>>>>>>>>>>>>> > currently. The >>>>>>>>>>>>>>>>> > C* docker images available are not made by us. >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > Kind Regards, >>>>>>>>>>>>>>>>> > Brandon >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > On Tue, Oct 24, 2023 at 3:31 PM Patrick McFadin >>>>>>>>>>>>>>>>> > <pmcfa...@gmail.com <mailto:pmcfa...@gmail.com>> wrote: >>>>>>>>>>>>>>>>> >> Let me make that really easy. Hell yes >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> >> Not everybody runs CCM, I've tried but I've met resistance. >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> >> Compiling your own version usually involves me saying the >>>>>>>>>>>>>>>>> >> words "Yes, ant realclean exists. I'm not trolling you" >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> >> docker pull <image> works on every OS and curates a single >>>>>>>>>>>>>>>>> >> node experience. >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> >> On Tue, Oct 24, 2023 at 12:37 PM Josh McKenzie >>>>>>>>>>>>>>>>> >> <jmcken...@apache.org <mailto:jmcken...@apache.org>> wrote: >>>>>>>>>>>>>>>>> >>> In order for the project to advertise the release outside >>>>>>>>>>>>>>>>> >>> the dev@ list it needs to be a formal release. >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> That's my reading as well: >>>>>>>>>>>>>>>>> >>> https://www.apache.org/legal/release-policy.html#release-definition >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> I wonder if there'd be value in us having a cronned job >>>>>>>>>>>>>>>>> >>> that'd do nightly docker container builds on trunk + >>>>>>>>>>>>>>>>> >>> feature branches, archived for N days, and we make that >>>>>>>>>>>>>>>>> >>> generally known to the dev@ list here so folks that want >>>>>>>>>>>>>>>>> >>> to poke at the current state of trunk or other branches >>>>>>>>>>>>>>>>> >>> could do so with very low friction. We'd probably see >>>>>>>>>>>>>>>>> >>> more engagement on feature branches if it was turn-key >>>>>>>>>>>>>>>>> >>> easy for other C* devs to spin the up and check them out. >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> For what you're talking about here Patrick (a docker >>>>>>>>>>>>>>>>> >>> image for folks outside the dev@ audience and more >>>>>>>>>>>>>>>>> >>> user-facing), we'd want to vote on it and go through the >>>>>>>>>>>>>>>>> >>> formal process. >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> On Tue, Oct 24, 2023, at 3:10 PM, Jeremiah Jordan wrote: >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> In order for the project to advertise the release outside >>>>>>>>>>>>>>>>> >>> the dev@ list it needs to be a formal release. That just >>>>>>>>>>>>>>>>> >>> means that there was a release vote and at least 3 PMC >>>>>>>>>>>>>>>>> >>> members +1’ed it, and there are more +1 than there are >>>>>>>>>>>>>>>>> >>> -1, and we follow all the normal release rules. The ASF >>>>>>>>>>>>>>>>> >>> release process doesn’t care what branch you cut the >>>>>>>>>>>>>>>>> >>> artifacts from or what version you call it. >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> So the project can cut artifacts for and release a >>>>>>>>>>>>>>>>> >>> 5.1-alpha1, 5.1-dev-preview1, what ever we want to >>>>>>>>>>>>>>>>> >>> version this thing, from trunk or any other branch name >>>>>>>>>>>>>>>>> >>> we want. >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> -Jeremiah >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> On Oct 24, 2023 at 2:03:41 PM, Patrick McFadin >>>>>>>>>>>>>>>>> >>> <pmcfa...@gmail.com <mailto:pmcfa...@gmail.com>> wrote: >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> I would like to have something for developers to use ASAP >>>>>>>>>>>>>>>>> >>> to try the Accord syntax. Very few people have seen it, >>>>>>>>>>>>>>>>> >>> and I think there's a learning curve we can start earlier. >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> It's my understanding that ASF policy is that it needs to >>>>>>>>>>>>>>>>> >>> be a project release to create a docker image. >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> On Tue, Oct 24, 2023 at 11:54 AM Jeremiah Jordan >>>>>>>>>>>>>>>>> >>> <jeremiah.jor...@gmail.com >>>>>>>>>>>>>>>>> >>> <mailto:jeremiah.jor...@gmail.com>> wrote: >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> If we decide to go the route of not merging TCM to the >>>>>>>>>>>>>>>>> >>> 5.0 branch. Do we actually need to immediately cut a 5.1 >>>>>>>>>>>>>>>>> >>> branch? Can we work on stabilizing things while it is in >>>>>>>>>>>>>>>>> >>> trunk and cut the 5.1 branch when we actually think we >>>>>>>>>>>>>>>>> >>> are near releasing? I don’t see any reason we can not >>>>>>>>>>>>>>>>> >>> cut “preview” artifacts from trunk? >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> -Jeremiah >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> On Oct 24, 2023 at 11:54:25 AM, Jon Haddad >>>>>>>>>>>>>>>>> >>> <rustyrazorbl...@apache.org >>>>>>>>>>>>>>>>> >>> <mailto:rustyrazorbl...@apache.org>> wrote: >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> I guess at the end of the day, shipping a release with a >>>>>>>>>>>>>>>>> >>> bunch of awesome features is better than holding it back. >>>>>>>>>>>>>>>>> >>> If there's 2 big releases in 6 months the community >>>>>>>>>>>>>>>>> >>> isn't any worse off. >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> We either ship something, or nothing, and something is >>>>>>>>>>>>>>>>> >>> probably better. >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> Jon >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> On 2023/10/24 16:27:04 Patrick McFadin wrote: >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> +1 to what you are saying, Josh. Based on the last >>>>>>>>>>>>>>>>> >>> survey, yes, everyone >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> was excited about Accord, but SAI and UCS were pretty >>>>>>>>>>>>>>>>> >>> high on the list. >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> Benedict and I had a good conversation last night, and >>>>>>>>>>>>>>>>> >>> now I understand >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> more essential details for this conversation. TCM is >>>>>>>>>>>>>>>>> >>> taking far more work >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> than initially scoped, and Accord depends on a stable >>>>>>>>>>>>>>>>> >>> TCM. TCM is months >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> behind and that's a critical fact, and one I personally >>>>>>>>>>>>>>>>> >>> just learned of. I >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> thought things were wrapping up this month, and we were >>>>>>>>>>>>>>>>> >>> in the testing >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> phase. I get why that's a topic we are dancing around. >>>>>>>>>>>>>>>>> >>> Nobody wants to say >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> ship dates are slipping because that's part of our >>>>>>>>>>>>>>>>> >>> culture. It's >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> disappointing and, if new information, an unwelcome >>>>>>>>>>>>>>>>> >>> surprise, but none of >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> us should be angry or in a blamey mood because I >>>>>>>>>>>>>>>>> >>> guarantee every one of us >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> has shipped the code late. My reaction yesterday was >>>>>>>>>>>>>>>>> >>> based on an incorrect >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> assumption. Now that I have a better picture, my point of >>>>>>>>>>>>>>>>> >>> view is changing. >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> Josh's point about what's best for users is crucial. >>>>>>>>>>>>>>>>> >>> Users deserve stable >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> code with a regular cadence of features that make their >>>>>>>>>>>>>>>>> >>> lives easier. If we >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> put 5.0 on hold for TCM + Accord, users will get neither >>>>>>>>>>>>>>>>> >>> for a very long >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> time. And I mentioned a disaster yesterday. A bigger >>>>>>>>>>>>>>>>> >>> disaster would be >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> shipping Accord with a major bug that causes data loss, >>>>>>>>>>>>>>>>> >>> eroding community >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> trust. Accord has to be the most bulletproof of all >>>>>>>>>>>>>>>>> >>> bulletproof features. >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> The pressure to ship is only going to increase and that's >>>>>>>>>>>>>>>>> >>> fertile ground >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> for that sort of bug. >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> So, taking a step back and with a clearer picture, I >>>>>>>>>>>>>>>>> >>> support the 5.0 + 5.1 >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> plan mainly because I don't think 5.1 is (or should be) a >>>>>>>>>>>>>>>>> >>> fast follow. >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> For the user community, the communication should be >>>>>>>>>>>>>>>>> >>> straightforward. TCM + >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> Accord are turning out to be much more complicated than >>>>>>>>>>>>>>>>> >>> was originally >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> scoped, and for good reasons. Our first principle is to >>>>>>>>>>>>>>>>> >>> provide a stable >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> and reliable system, so as a result, we'll be de-coupling >>>>>>>>>>>>>>>>> >>> TCM + Accord from >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> 5.0 into a 5.1 branch, which is available in parallel to >>>>>>>>>>>>>>>>> >>> 5.0 while >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> additional hardening and testing is done. We can >>>>>>>>>>>>>>>>> >>> communicate this in a blog >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> post., >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> To make this much more palatable to our use community, if >>>>>>>>>>>>>>>>> >>> we can get a >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> build and docker image available ASAP with Accord, it >>>>>>>>>>>>>>>>> >>> will allow developers >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> to start playing with the syntax. Up to this point, that >>>>>>>>>>>>>>>>> >>> hasn't been widely >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> available unless you compile the code yourself. >>>>>>>>>>>>>>>>> >>> Developers need to >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> understand how this will work in an application, and up >>>>>>>>>>>>>>>>> >>> to this point, the >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> syntax is text they see in my slides. We need to get some >>>>>>>>>>>>>>>>> >>> hands-on and that >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> will get our user community engaged on Accord this >>>>>>>>>>>>>>>>> >>> calendar year. The >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> feedback may even uncover some critical changes we'll >>>>>>>>>>>>>>>>> >>> need to make. Lack of >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> access to Accord by developers is a critical problem we >>>>>>>>>>>>>>>>> >>> can fix soon and >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> there will be plenty of excitement there and start >>>>>>>>>>>>>>>>> >>> building use cases >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> before the final code ships. >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> I'm bummed but realistic. It sucks that I won't have a >>>>>>>>>>>>>>>>> >>> pony for Christmas, >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> but maybe one for my birthday? >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> Patrick >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> On Tue, Oct 24, 2023 at 7:23 AM Josh McKenzie >>>>>>>>>>>>>>>>> >>> <jmcken...@apache.org <mailto:jmcken...@apache.org>> >>>>>>>>>>>>>>>>> >>> wrote: >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>>> Maybe it won't be a glamorous release but shipping >>>>>>>>>>>>>>>>> >>>> 5.0 mitigates our worst case scenario. >>>>>>>>>>>>>>>>> >>>> I disagree with this characterization of 5.0 personally. >>>>>>>>>>>>>>>>> >>>> UCS, SAI, Trie >>>>>>>>>>>>>>>>> >>>> memtables and sstables, maybe vector ANN if the >>>>>>>>>>>>>>>>> >>>> sub-tasks on C-18715 are >>>>>>>>>>>>>>>>> >>>> accurate, all combine to make 5.0 a pretty glamorous >>>>>>>>>>>>>>>>> >>>> release IMO >>>>>>>>>>>>>>>>> >>>> independent of TCM and Accord. Accord is a true >>>>>>>>>>>>>>>>> >>>> paradigm-shift game-changer >>>>>>>>>>>>>>>>> >>>> so it's easy to think of 5.0 as uneventful in >>>>>>>>>>>>>>>>> >>>> comparison, and TCM helps >>>>>>>>>>>>>>>>> >>>> resolve one of the biggest pain-points in our system for >>>>>>>>>>>>>>>>> >>>> over a decade, but >>>>>>>>>>>>>>>>> >>>> I think 5.0 is a very meaty release in its own right >>>>>>>>>>>>>>>>> >>>> today. >>>>>>>>>>>>>>>>> >>>> Anyway - I agree with you Brandon re: timelines. If >>>>>>>>>>>>>>>>> >>>> things take longer >>>>>>>>>>>>>>>>> >>>> than we'd hope (which, if I think back, they do roughly >>>>>>>>>>>>>>>>> >>>> 100% of the time on >>>>>>>>>>>>>>>>> >>>> this project), blocking on these features could both >>>>>>>>>>>>>>>>> >>>> lead to a significant >>>>>>>>>>>>>>>>> >>>> delay in 5.0 going out as well as increasing pressure >>>>>>>>>>>>>>>>> >>>> and risk of burnout >>>>>>>>>>>>>>>>> >>>> on the folks working on it. While I believe we all need >>>>>>>>>>>>>>>>> >>>> some balanced >>>>>>>>>>>>>>>>> >>>> urgency to do our best work, being under the gun for >>>>>>>>>>>>>>>>> >>>> something with a hard >>>>>>>>>>>>>>>>> >>>> deadline or having an entire project drag along blocked >>>>>>>>>>>>>>>>> >>>> on you is not where >>>>>>>>>>>>>>>>> >>>> I want any of us to be. >>>>>>>>>>>>>>>>> >>>> Part of why we talked about going to primarily annual >>>>>>>>>>>>>>>>> >>>> calendar-based >>>>>>>>>>>>>>>>> >>>> releases was to avoid precisely this situation, where >>>>>>>>>>>>>>>>> >>>> something that >>>>>>>>>>>>>>>>> >>>> *feels* right at the cusp of merging leads us to delay a >>>>>>>>>>>>>>>>> >>>> release >>>>>>>>>>>>>>>>> >>>> repeatedly. We discussed this a couple times this year: >>>>>>>>>>>>>>>>> >>>> 1: >>>>>>>>>>>>>>>>> >>>> https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3, >>>>>>>>>>>>>>>>> >>>> where Mick proposed a "soft-freeze" for everything w/out >>>>>>>>>>>>>>>>> >>>> exception and 1st >>>>>>>>>>>>>>>>> >>>> week October "hard-freeze", and there was assumed to be >>>>>>>>>>>>>>>>> >>>> lazy consensus >>>>>>>>>>>>>>>>> >>>> 2: >>>>>>>>>>>>>>>>> >>>> https://lists.apache.org/thread/mzj3dq8b7mzf60k6mkby88b9n9ywmsgw, >>>>>>>>>>>>>>>>> >>>> where we kept along with what we discussed in 1 but >>>>>>>>>>>>>>>>> >>>> added in CEP-30 to be >>>>>>>>>>>>>>>>> >>>> waivered in as well. >>>>>>>>>>>>>>>>> >>>> So. We're at a crossroads here where we need to either >>>>>>>>>>>>>>>>> >>>> follow through with >>>>>>>>>>>>>>>>> >>>> what we all agreed to earlier this year, or acknowledge >>>>>>>>>>>>>>>>> >>>> that our best >>>>>>>>>>>>>>>>> >>>> intention of calendar-based releases can't stand up to >>>>>>>>>>>>>>>>> >>>> our optimism and >>>>>>>>>>>>>>>>> >>>> desire to get these features into the next major. >>>>>>>>>>>>>>>>> >>>> There's no immediate obvious better path to me in terms >>>>>>>>>>>>>>>>> >>>> of what's best for >>>>>>>>>>>>>>>>> >>>> our users. This is a situation of risk tolerance with a >>>>>>>>>>>>>>>>> >>>> lot of unknowns >>>>>>>>>>>>>>>>> >>>> that could go either way. >>>>>>>>>>>>>>>>> >>>> Any light that folks active on TCM and Accord could shed >>>>>>>>>>>>>>>>> >>>> in terms of their >>>>>>>>>>>>>>>>> >>>> best and worst-case scenarios on timelines for those >>>>>>>>>>>>>>>>> >>>> features might help us >>>>>>>>>>>>>>>>> >>>> narrow this down a bit. Otherwise, I'm inclined to defer >>>>>>>>>>>>>>>>> >>>> to our past selves >>>>>>>>>>>>>>>>> >>>> and fall back to "we agreed to yearly calendar releases >>>>>>>>>>>>>>>>> >>>> for good reason. >>>>>>>>>>>>>>>>> >>>> Let's stick to our guns." >>>>>>>>>>>>>>>>> >>>> On Tue, Oct 24, 2023, at 9:37 AM, Brandon Williams wrote: >>>>>>>>>>>>>>>>> >>>> The concern I have with this is that is a big slippery >>>>>>>>>>>>>>>>> >>>> 'if' that >>>>>>>>>>>>>>>>> >>>> involves development time estimation, and if it tends to >>>>>>>>>>>>>>>>> >>>> take longer >>>>>>>>>>>>>>>>> >>>> than we estimate (as these things tend to do), then I >>>>>>>>>>>>>>>>> >>>> can see a future >>>>>>>>>>>>>>>>> >>>> where 5.0 is delayed until the middle of 2024, and I >>>>>>>>>>>>>>>>> >>>> really don't want >>>>>>>>>>>>>>>>> >>>> that to happen. Maybe it won't be a glamorous release >>>>>>>>>>>>>>>>> >>>> but shipping >>>>>>>>>>>>>>>>> >>>> 5.0 mitigates our worst case scenario. >>>>>>>>>>>>>>>>> >>>> Kind Regards, >>>>>>>>>>>>>>>>> >>>> Brandon >>>>>>>>>>>>>>>>> >>>> On Mon, Oct 23, 2023 at 4:02 PM Dinesh Joshi >>>>>>>>>>>>>>>>> >>>> <djo...@apache.org <mailto:djo...@apache.org>> wrote: >>>>>>>>>>>>>>>>> >>>>> I have a strong preference to move out the 5.0 date to >>>>>>>>>>>>>>>>> >>>>> have accord and >>>>>>>>>>>>>>>>> >>>> TCM. I don’t see the point in shipping 5.0 without these >>>>>>>>>>>>>>>>> >>>> features >>>>>>>>>>>>>>>>> >>>> especially if 5.1 is going to follow close behind it. >>>>>>>>>>>>>>>>> >>>>> Dinesh >>>>>>>>>>>>>>>>> >>>>> On Oct 23, 2023, at 4:52 AM, Mick Semb Wever >>>>>>>>>>>>>>>>> >>>>> <m...@apache.org <mailto:m...@apache.org>> wrote: >>>>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>>>> >>>>> The TCM work (CEP-21) is in its review stage but being >>>>>>>>>>>>>>>>> >>>>> well past our >>>>>>>>>>>>>>>>> >>>> cut-off date¹ for merging, and now jeopardising 5.0 GA >>>>>>>>>>>>>>>>> >>>> efforts, I would >>>>>>>>>>>>>>>>> >>>> like to propose the following. >>>>>>>>>>>>>>>>> >>>>> We merge TCM and Accord only to trunk. Then branch >>>>>>>>>>>>>>>>> >>>>> cassandra-5.1 and >>>>>>>>>>>>>>>>> >>>> cut an immediate 5.1-alpha1 release. >>>>>>>>>>>>>>>>> >>>>> I see this as a win-win scenario for us, considering >>>>>>>>>>>>>>>>> >>>>> our current >>>>>>>>>>>>>>>>> >>>> situation. (Though it is unfortunate that Accord is >>>>>>>>>>>>>>>>> >>>> included in this >>>>>>>>>>>>>>>>> >>>> scenario because we agreed it to be based upon TCM.) >>>>>>>>>>>>>>>>> >>>>> This will mean… >>>>>>>>>>>>>>>>> >>>>> - We get to focus on getting 5.0 to beta and GA, >>>>>>>>>>>>>>>>> >>>>> which already has a >>>>>>>>>>>>>>>>> >>>> ton of features users want. >>>>>>>>>>>>>>>>> >>>>> - We get an alpha release with TCM and Accord into >>>>>>>>>>>>>>>>> >>>>> users hands quickly >>>>>>>>>>>>>>>>> >>>> for broader testing and feedback. >>>>>>>>>>>>>>>>> >>>>> - We isolate GA efforts on TCM and Accord – giving >>>>>>>>>>>>>>>>> >>>>> oss and downstream >>>>>>>>>>>>>>>>> >>>> engineers time and patience reviewing and testing. TCM >>>>>>>>>>>>>>>>> >>>> will be the biggest >>>>>>>>>>>>>>>>> >>>> patch ever to land in C*. >>>>>>>>>>>>>>>>> >>>>> - Give users a choice for a more incremental upgrade >>>>>>>>>>>>>>>>> >>>>> approach, given >>>>>>>>>>>>>>>>> >>>> just how many new features we're putting on them in one >>>>>>>>>>>>>>>>> >>>> year. >>>>>>>>>>>>>>>>> >>>>> - 5.1 w/ TCM and Accord will maintain its upgrade >>>>>>>>>>>>>>>>> >>>>> compatibility with >>>>>>>>>>>>>>>>> >>>> all 4.x versions, just as if it had landed in 5.0. >>>>>>>>>>>>>>>>> >>>>> The risks/costs this introduces are >>>>>>>>>>>>>>>>> >>>>> - If we cannot stabilise TCM and/or Accord on the >>>>>>>>>>>>>>>>> >>>>> cassandra-5.1 branch, >>>>>>>>>>>>>>>>> >>>> and at some point decide to undo this work, while we can >>>>>>>>>>>>>>>>> >>>> throw away the >>>>>>>>>>>>>>>>> >>>> cassandra-5.1 branch we would need to do a bit of work >>>>>>>>>>>>>>>>> >>>> reverting the >>>>>>>>>>>>>>>>> >>>> changes in trunk. This is a _very_ edge case, as >>>>>>>>>>>>>>>>> >>>> confidence levels on the >>>>>>>>>>>>>>>>> >>>> design and implementation of both are already tested and >>>>>>>>>>>>>>>>> >>>> high. >>>>>>>>>>>>>>>>> >>>>> - We will have to maintain an additional branch. I >>>>>>>>>>>>>>>>> >>>>> propose that we >>>>>>>>>>>>>>>>> >>>> treat the 5.1 branch in the same maintenance window as >>>>>>>>>>>>>>>>> >>>> 5.0 (like we have >>>>>>>>>>>>>>>>> >>>> with 3.0 and 3.11). This also adds the merge path >>>>>>>>>>>>>>>>> >>>> overhead. >>>>>>>>>>>>>>>>> >>>>> - Reviewing of TCM and Accord will continue to happen >>>>>>>>>>>>>>>>> >>>>> post-merge. This >>>>>>>>>>>>>>>>> >>>> is not our normal practice, but this work will have >>>>>>>>>>>>>>>>> >>>> already received its >>>>>>>>>>>>>>>>> >>>> two +1s from committers, and such ongoing review effort >>>>>>>>>>>>>>>>> >>>> is akin to GA >>>>>>>>>>>>>>>>> >>>> stabilisation work on release branches. >>>>>>>>>>>>>>>>> >>>>> I see no other ok solution in front of us that gets us >>>>>>>>>>>>>>>>> >>>>> at least both the >>>>>>>>>>>>>>>>> >>>> 5.0 beta and TCM+Accord alpha releases this year. >>>>>>>>>>>>>>>>> >>>> Keeping in mind users >>>>>>>>>>>>>>>>> >>>> demand to start experimenting with these features, and >>>>>>>>>>>>>>>>> >>>> our Cassandra Summit >>>>>>>>>>>>>>>>> >>>> in December. >>>>>>>>>>>>>>>>> >>>>> 1) >>>>>>>>>>>>>>>>> >>>>> https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3 >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> >>>
