The proposal includes 3 things: 1. Do not include TCM and Accord in 5.0 to avoid delaying 5.0 2. The next release will be 5.1 and will include only Accord and TCM 3. Merge TCM and Accord right now in 5.1 (making an initial release)
I am fine with question 1 and do not have a strong opinion on which way to go. 2. Means that every new feature will have to wait for post 5.1 even if it is ready before 5.1 is stabilized and shipped. If we do a 5.1 release why not take it as an opportunity to release more things. I am not saying that we will. Just that we should let that door open. 3. There is a need to merge TCM and Accord as maintaining those separate branches is costly in terms of time and energy. I fully understand that. On the other hand merging TCM and Accord will make the TCM review harder and I do believe that this second round of review is valuable as it already uncovered a valid issue. Nevertheless, I am fine with merging TCM as soon as it passes CI and continuing the review after the merge. If we cannot meet at least that quality level (Green CI) we should not merge just for creating an 5.1.alpha release for the summit. Now, I am totally fine with a preview release without numbering and with big warnings that will only serve as a preview for the summit. Le mer. 25 oct. 2023 à 06:33, Berenguer Blasi <berenguerbl...@gmail.com> a écrit : > I also think there's many good new features in 5.0 already they'd make a > good release even on their own. My 2 cts. > > On 24/10/23 23:20, Brandon Williams wrote: > > The catch here is that we don't publish docker images currently. The > > C* docker images available are not made by us. > > > > Kind Regards, > > Brandon > > > > On Tue, Oct 24, 2023 at 3:31 PM Patrick McFadin <pmcfa...@gmail.com> > wrote: > >> Let me make that really easy. Hell yes > >> > >> Not everybody runs CCM, I've tried but I've met resistance. > >> > >> Compiling your own version usually involves me saying the words "Yes, > ant realclean exists. I'm not trolling you" > >> > >> docker pull <image> works on every OS and curates a single node > experience. > >> > >> > >> > >> On Tue, Oct 24, 2023 at 12:37 PM Josh McKenzie <jmcken...@apache.org> > wrote: > >>> In order for the project to advertise the release outside the dev@ > list it needs to be a formal release. > >>> > >>> That's my reading as well: > >>> https://www.apache.org/legal/release-policy.html#release-definition > >>> > >>> I wonder if there'd be value in us having a cronned job that'd do > nightly docker container builds on trunk + feature branches, archived for N > days, and we make that generally known to the dev@ list here so folks > that want to poke at the current state of trunk or other branches could do > so with very low friction. We'd probably see more engagement on feature > branches if it was turn-key easy for other C* devs to spin the up and check > them out. > >>> > >>> For what you're talking about here Patrick (a docker image for folks > outside the dev@ audience and more user-facing), we'd want to vote on it > and go through the formal process. > >>> > >>> On Tue, Oct 24, 2023, at 3:10 PM, Jeremiah Jordan wrote: > >>> > >>> In order for the project to advertise the release outside the dev@ > list it needs to be a formal release. That just means that there was a > release vote and at least 3 PMC members +1’ed it, and there are more +1 > than there are -1, and we follow all the normal release rules. The ASF > release process doesn’t care what branch you cut the artifacts from or what > version you call it. > >>> > >>> So the project can cut artifacts for and release a 5.1-alpha1, > 5.1-dev-preview1, what ever we want to version this thing, from trunk or > any other branch name we want. > >>> > >>> -Jeremiah > >>> > >>> On Oct 24, 2023 at 2:03:41 PM, Patrick McFadin <pmcfa...@gmail.com> > wrote: > >>> > >>> I would like to have something for developers to use ASAP to try the > Accord syntax. Very few people have seen it, and I think there's a learning > curve we can start earlier. > >>> > >>> It's my understanding that ASF policy is that it needs to be a project > release to create a docker image. > >>> > >>> On Tue, Oct 24, 2023 at 11:54 AM Jeremiah Jordan < > jeremiah.jor...@gmail.com> wrote: > >>> > >>> If we decide to go the route of not merging TCM to the 5.0 branch. Do > we actually need to immediately cut a 5.1 branch? Can we work on > stabilizing things while it is in trunk and cut the 5.1 branch when we > actually think we are near releasing? I don’t see any reason we can not > cut “preview” artifacts from trunk? > >>> > >>> -Jeremiah > >>> > >>> On Oct 24, 2023 at 11:54:25 AM, Jon Haddad <rustyrazorbl...@apache.org> > wrote: > >>> > >>> I guess at the end of the day, shipping a release with a bunch of > awesome features is better than holding it back. If there's 2 big releases > in 6 months the community isn't any worse off. > >>> > >>> We either ship something, or nothing, and something is probably better. > >>> > >>> Jon > >>> > >>> > >>> On 2023/10/24 16:27:04 Patrick McFadin wrote: > >>> > >>> +1 to what you are saying, Josh. Based on the last survey, yes, > everyone > >>> > >>> was excited about Accord, but SAI and UCS were pretty high on the list. > >>> > >>> > >>> Benedict and I had a good conversation last night, and now I understand > >>> > >>> more essential details for this conversation. TCM is taking far more > work > >>> > >>> than initially scoped, and Accord depends on a stable TCM. TCM is > months > >>> > >>> behind and that's a critical fact, and one I personally just learned > of. I > >>> > >>> thought things were wrapping up this month, and we were in the testing > >>> > >>> phase. I get why that's a topic we are dancing around. Nobody wants to > say > >>> > >>> ship dates are slipping because that's part of our culture. It's > >>> > >>> disappointing and, if new information, an unwelcome surprise, but none > of > >>> > >>> us should be angry or in a blamey mood because I guarantee every one > of us > >>> > >>> has shipped the code late. My reaction yesterday was based on an > incorrect > >>> > >>> assumption. Now that I have a better picture, my point of view is > changing. > >>> > >>> > >>> Josh's point about what's best for users is crucial. Users deserve > stable > >>> > >>> code with a regular cadence of features that make their lives easier. > If we > >>> > >>> put 5.0 on hold for TCM + Accord, users will get neither for a very > long > >>> > >>> time. And I mentioned a disaster yesterday. A bigger disaster would be > >>> > >>> shipping Accord with a major bug that causes data loss, eroding > community > >>> > >>> trust. Accord has to be the most bulletproof of all bulletproof > features. > >>> > >>> The pressure to ship is only going to increase and that's fertile > ground > >>> > >>> for that sort of bug. > >>> > >>> > >>> So, taking a step back and with a clearer picture, I support the 5.0 + > 5.1 > >>> > >>> plan mainly because I don't think 5.1 is (or should be) a fast follow. > >>> > >>> > >>> For the user community, the communication should be straightforward. > TCM + > >>> > >>> Accord are turning out to be much more complicated than was originally > >>> > >>> scoped, and for good reasons. Our first principle is to provide a > stable > >>> > >>> and reliable system, so as a result, we'll be de-coupling TCM + Accord > from > >>> > >>> 5.0 into a 5.1 branch, which is available in parallel to 5.0 while > >>> > >>> additional hardening and testing is done. We can communicate this in a > blog > >>> > >>> post., > >>> > >>> > >>> To make this much more palatable to our use community, if we can get a > >>> > >>> build and docker image available ASAP with Accord, it will allow > developers > >>> > >>> to start playing with the syntax. Up to this point, that hasn't been > widely > >>> > >>> available unless you compile the code yourself. Developers need to > >>> > >>> understand how this will work in an application, and up to this point, > the > >>> > >>> syntax is text they see in my slides. We need to get some hands-on and > that > >>> > >>> will get our user community engaged on Accord this calendar year. The > >>> > >>> feedback may even uncover some critical changes we'll need to make. > Lack of > >>> > >>> access to Accord by developers is a critical problem we can fix soon > and > >>> > >>> there will be plenty of excitement there and start building use cases > >>> > >>> before the final code ships. > >>> > >>> > >>> I'm bummed but realistic. It sucks that I won't have a pony for > Christmas, > >>> > >>> but maybe one for my birthday? > >>> > >>> > >>> Patrick > >>> > >>> > >>> > >>> > >>> On Tue, Oct 24, 2023 at 7:23 AM Josh McKenzie <jmcken...@apache.org> > wrote: > >>> > >>> > >>>> Maybe it won't be a glamorous release but shipping > >>>> 5.0 mitigates our worst case scenario. > >>>> I disagree with this characterization of 5.0 personally. UCS, SAI, > Trie > >>>> memtables and sstables, maybe vector ANN if the sub-tasks on C-18715 > are > >>>> accurate, all combine to make 5.0 a pretty glamorous release IMO > >>>> independent of TCM and Accord. Accord is a true paradigm-shift > game-changer > >>>> so it's easy to think of 5.0 as uneventful in comparison, and TCM > helps > >>>> resolve one of the biggest pain-points in our system for over a > decade, but > >>>> I think 5.0 is a very meaty release in its own right today. > >>>> Anyway - I agree with you Brandon re: timelines. If things take longer > >>>> than we'd hope (which, if I think back, they do roughly 100% of the > time on > >>>> this project), blocking on these features could both lead to a > significant > >>>> delay in 5.0 going out as well as increasing pressure and risk of > burnout > >>>> on the folks working on it. While I believe we all need some balanced > >>>> urgency to do our best work, being under the gun for something with a > hard > >>>> deadline or having an entire project drag along blocked on you is not > where > >>>> I want any of us to be. > >>>> Part of why we talked about going to primarily annual calendar-based > >>>> releases was to avoid precisely this situation, where something that > >>>> *feels* right at the cusp of merging leads us to delay a release > >>>> repeatedly. We discussed this a couple times this year: > >>>> 1: https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3, > >>>> where Mick proposed a "soft-freeze" for everything w/out exception > and 1st > >>>> week October "hard-freeze", and there was assumed to be lazy consensus > >>>> 2: https://lists.apache.org/thread/mzj3dq8b7mzf60k6mkby88b9n9ywmsgw, > >>>> where we kept along with what we discussed in 1 but added in CEP-30 > to be > >>>> waivered in as well. > >>>> So. We're at a crossroads here where we need to either follow through > with > >>>> what we all agreed to earlier this year, or acknowledge that our best > >>>> intention of calendar-based releases can't stand up to our optimism > and > >>>> desire to get these features into the next major. > >>>> There's no immediate obvious better path to me in terms of what's > best for > >>>> our users. This is a situation of risk tolerance with a lot of > unknowns > >>>> that could go either way. > >>>> Any light that folks active on TCM and Accord could shed in terms of > their > >>>> best and worst-case scenarios on timelines for those features might > help us > >>>> narrow this down a bit. Otherwise, I'm inclined to defer to our past > selves > >>>> and fall back to "we agreed to yearly calendar releases for good > reason. > >>>> Let's stick to our guns." > >>>> On Tue, Oct 24, 2023, at 9:37 AM, Brandon Williams wrote: > >>>> The concern I have with this is that is a big slippery 'if' that > >>>> involves development time estimation, and if it tends to take longer > >>>> than we estimate (as these things tend to do), then I can see a future > >>>> where 5.0 is delayed until the middle of 2024, and I really don't want > >>>> that to happen. Maybe it won't be a glamorous release but shipping > >>>> 5.0 mitigates our worst case scenario. > >>>> Kind Regards, > >>>> Brandon > >>>> On Mon, Oct 23, 2023 at 4:02 PM Dinesh Joshi <djo...@apache.org> > wrote: > >>>>> I have a strong preference to move out the 5.0 date to have accord > and > >>>> TCM. I don’t see the point in shipping 5.0 without these features > >>>> especially if 5.1 is going to follow close behind it. > >>>>> Dinesh > >>>>> On Oct 23, 2023, at 4:52 AM, Mick Semb Wever <m...@apache.org> wrote: > >>>>> > >>>>> The TCM work (CEP-21) is in its review stage but being well past our > >>>> cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I > would > >>>> like to propose the following. > >>>>> We merge TCM and Accord only to trunk. Then branch cassandra-5.1 and > >>>> cut an immediate 5.1-alpha1 release. > >>>>> I see this as a win-win scenario for us, considering our current > >>>> situation. (Though it is unfortunate that Accord is included in this > >>>> scenario because we agreed it to be based upon TCM.) > >>>>> This will mean… > >>>>> - We get to focus on getting 5.0 to beta and GA, which already has > a > >>>> ton of features users want. > >>>>> - We get an alpha release with TCM and Accord into users hands > quickly > >>>> for broader testing and feedback. > >>>>> - We isolate GA efforts on TCM and Accord – giving oss and > downstream > >>>> engineers time and patience reviewing and testing. TCM will be the > biggest > >>>> patch ever to land in C*. > >>>>> - Give users a choice for a more incremental upgrade approach, > given > >>>> just how many new features we're putting on them in one year. > >>>>> - 5.1 w/ TCM and Accord will maintain its upgrade compatibility > with > >>>> all 4.x versions, just as if it had landed in 5.0. > >>>>> The risks/costs this introduces are > >>>>> - If we cannot stabilise TCM and/or Accord on the cassandra-5.1 > branch, > >>>> and at some point decide to undo this work, while we can throw away > the > >>>> cassandra-5.1 branch we would need to do a bit of work reverting the > >>>> changes in trunk. This is a _very_ edge case, as confidence levels > on the > >>>> design and implementation of both are already tested and high. > >>>>> - We will have to maintain an additional branch. I propose that we > >>>> treat the 5.1 branch in the same maintenance window as 5.0 (like we > have > >>>> with 3.0 and 3.11). This also adds the merge path overhead. > >>>>> - Reviewing of TCM and Accord will continue to happen post-merge. > This > >>>> is not our normal practice, but this work will have already received > its > >>>> two +1s from committers, and such ongoing review effort is akin to GA > >>>> stabilisation work on release branches. > >>>>> I see no other ok solution in front of us that gets us at least both > the > >>>> 5.0 beta and TCM+Accord alpha releases this year. Keeping in mind > users > >>>> demand to start experimenting with these features, and our Cassandra > Summit > >>>> in December. > >>>>> 1) https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3 > >>> > >>> >
