Re: [DISCUSS] 5.1 should be 6.0

> A lot has already been cleaned in 4.0 that makes 3.x to 5.x now non-functioning. And the cleaner code certainly helps navigate the code base quicker. Sure - I was using that as an example, looking back in hindsight. Like, if today people could go from 2.1 -> 5.0 it would be cool. Applying it

Re: [DISCUSS] 5.1 should be 6.0

I know a few teams on 2.2 that would *love* to be able to jump right to > 5.0. Once you fall far enough behind, upgrading to another version that's > already deprecated becomes paralyzing. I don't expect 2.2 compatibility > btw, just using it as an example. > > If it can be done, it would make a

Re: [DISCUSS] 5.1 should be 6.0

I have to agree. Harder upgrades are simpler for users? No chance.I like Jeremiah’s proposal, but would also be happy with a less rigid “we support as many versions as we can without it becoming a burden” - which for some versions might be one minor/major, and for others might be three or more.On 1

Re: [DISCUSS] 5.1 should be 6.0

I know a few teams on 2.2 that would *love* to be able to jump right to 5.0. Once you fall far enough behind, upgrading to another version that's already deprecated becomes paralyzing. I don't expect 2.2 compatibility btw, just using it as an example. If it can be done, it would make a lot of fo

Re: [DISCUSS] 5.1 should be 6.0

I think the simplest upgrade path is actually “you can upgrade from any currently ’supported’ version to this new one”. So for 5.1/6.0 that would be 4.0/4.1/5.0. That has always been my preferred support model for upgrades. -Jeremiah On Dec 10, 2024 at 3:45:52 PM, Mick Semb Wever wrote: > >

Re: [DISCUSS] 5.1 should be 6.0

I would very much rather keep our upgrade paths simple and intuitive: You can only upgrade between adjacent major versions. This does catch users, and keeping it simple has helped a lot. I find it helps internally working on the code too, with a focus on stability for online upgrades, which does q

Re: [DISCUSS] 5.1 should be 6.0

Usually, a major release would bump the Java and Python supported versions. Both Java and Python are on well-published and faster release cycles. On Tue, Dec 10, 2024 at 3:40 PM Paulo Motta wrote: > I share this sentiment. Outside of marketing and API compatibility > considerations, I think the

Re: [DISCUSS] 5.1 should be 6.0

The only thing the version numbers communicate for upgrades is a minimum guarantee. Just as we allowed 3.x to 5.0 we can and should allow 4.x to 6.0. If anything I would like to see us move towards a norm of supporting upgrade across more milestone releases as standard.On 10 Dec 2024, at 20:16, Cal

Re: [DISCUSS] 5.1 should be 6.0

I share this sentiment. Outside of marketing and API compatibility considerations, I think the changes are significant enough to warrant a major version bump, since it represents a new generation of the database. On Tue, Dec 10, 2024 at 1:02 PM Brandon Williams wrote: > Even if TCM is api-compat

Re: [DISCUSS] 5.1 should be 6.0

I don't care what we call it as long as 4.1 -> 5.1/6.0 upgrades are possible. On Tue, Dec 10, 2024 at 1:28 PM David Capwell wrote: > Given our version support… if we do make this change does this imply users > must do the following to get to 6.0? > > 4.x upgrade to 5.0 > 5.0 upgrade to 6.0 > > S

Re: [DISCUSS] 5.1 should be 6.0

Given our version support… if we do make this change does this imply users must do the following to get to 6.0? 4.x upgrade to 5.0 5.0 upgrade to 6.0 So no 4.x to 6.0? Given that this is 5.1 atm we are expected to support 4.x to 5.1 upgrades, but switching to 6.0 that isn’t true based off our

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

So some questions to test a world w/3 classifications (Preview, Beta, GA): - What would we do with the current experimental features (MV's, JDK17, witnesses, etc)? Flag them as preview or beta as appropriate on a case-by-case basis and add runtime warnings / documentation where missing? - What w

Re: [DISCUSS] 5.1 should be 6.0

> This is another topic we basically revisit afresh every time :) > > I think it’s fine to bump for marketing or vibe reasons, I would support it. > I don’t think we need to confect some weak semverish justification. Vibes ftw. Lets see if any strong contradicting opinions pop up on here in the

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

Yep, I agree with this - we can revisit if we ever absolutely feel the need to add additional states for exceptional circumstances. > On 10 Dec 2024, at 13:24, Patrick McFadin wrote: > > -1 on unstable. It's way too many words than are needed. Three is a > magic number and fits: > > Preview >

Re: [DISCUSS] 5.1 should be 6.0

I was waiting for this discussion to happen again. :p What you call marketing, I call end-user communication. I'll leave this open question, but what do we want to communicate to the user base, and how should they approach this new feature set? Patrick On Tue, Dec 10, 2024 at 10:10 AM Benedict E

Re: [DISCUSS] 5.1 should be 6.0

6.0 even just for the reasons Brandon mentioned sounds reasonable to me. And I acknowledge there are more On Tue, 10 Dec 2024 at 13:10, Benedict Elliott Smith wrote: > This is another topic we basically revisit afresh every time :) > > I think it’s fine to bump for marketing or vibe reasons, I w

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

-1 on unstable. It's way too many words than are needed. Three is a magic number and fits: Preview Beta GA As a matter of testing the process, any pending CEP should go though this exercise so we can see how it will work. PS Got the actual numbers from Whimsy. DEV - 1425 users USER - 2650 This

Re: [DISCUSS] 5.1 should be 6.0

This is another topic we basically revisit afresh every time :) I think it’s fine to bump for marketing or vibe reasons, I would support it. I don’t think we need to confect some weak semverish justification. > On 10 Dec 2024, at 13:01, Brandon Williams wrote: > > Even if TCM is api-compatibl

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

As an aside, it would be nice to admit we basically revisit everything each time it becomes relevant again, and for policy decisions like this (that don’t need to be agreed in advance) we should try to legislate the minimum necessary policy to proceed today, and leave future refinements for late

Re: [DISCUSS] 5.1 should be 6.0

Even if TCM is api-compatible, it will change how operators run Cassandra in a significant way (like, different procedures from every previous version.) I think that justifies a major. Kind Regards, Brandon On Tue, Dec 10, 2024 at 11:51 AM Jeff Jirsa wrote: > > You’ve added a ton of API surface

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

I agree with Aleksey that if we think something is broken, we shouldn’t use euphemisms, and for this reason I don’t like unstable (this could for instance simply mean API unstable). If we intend to never need this descriptor, we should avoid bike-shedding and insert a “placeholder” for now to be

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

What JD and Josh said resonates also with the thoughts I had. On Tue, 10 Dec 2024 at 12:46, Caleb Rackliffe wrote: > +1 to Josh's refinement of JD's proposal > > On Tue, Dec 10, 2024 at 11:42 AM Josh McKenzie > wrote: > >> +1 to this classification with one addition. I think we need to augment

Re: [DISCUSS] 5.1 should be 6.0

You’ve added a ton of API surface to transaction behavior and cluster management. The TCM may or may not be strictly breaking, but they’re fundamentally very very different, so even with semver as the only standard, I think you can justify a major. But also, let’s just acknowledge that marketin

Re: [DISCUSS] 5.1 should be 6.0

Currently we reserve MAJOR in semver changes for API breaking only: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199530302#Patching,versioning,andLTSreleases-Versioningandtargeting: That's consistent w/semver itself: link : > Given

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

+1 to Josh's refinement of JD's proposal On Tue, Dec 10, 2024 at 11:42 AM Josh McKenzie wrote: > +1 to this classification with one addition. I think we need to augment > this with formalization on what we do with features we don't recommend > people use (i.e. MV in their current incarnation). F

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

+1 to this classification with one addition. I think we need to augment this with formalization on what we do with features we don't recommend people use (i.e. MV in their current incarnation). For something retroactively found to be unstable, we could add an "Unstable" qualification for it, lea

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

Yep, I agree with all of this line of discussion. +1 any reasonable variation of Aleksey, Patrick and Jeremiah’s proposals. > On 10 Dec 2024, at 12:29, Jeremiah Jordan wrote: > > I agree with Aleksey and Patrick. We should define terminology and then > stick to it. My preferred list would be

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

Let's say we went with the preview -> beta -> GA option. Does something like SASI stay in "experimental" while MV, transient replication, etc. move to "preview"? On Tue, Dec 10, 2024 at 11:30 AM Jeremiah Jordan wrote: > I agree with Aleksey and Patrick. We should define terminology and then > s

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

I agree with Aleksey and Patrick. We should define terminology and then stick to it. My preferred list would be: 1. Preview - Ready to be tried by end users but has caveats and most likely is not api stable. 2. Beta - Feature complete/API stable but has not had enough testing to be

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

I'm going to try to pull this back from the inevitable bikeshedding and airing of grievances that happen. Rewind all the way back to Josh's original point, which is a defined process. Why I really love this being brought up is our maturing process of communicating to the larger user base. The dev

Re: [DISCUSS] 5.1 should be 6.0

The question is if we are signaling compatibility or purely marketing with the release number. We dropped compatibility with a few things in 5.0, which was the reason for the .0 rather than 4.2. I don’t know if we are breaking any compatibility with current trunk? Though maybe some of the TCM st

[DISCUSS] 5.1 should be 6.0

Keeping this short. I'm not sure why we're calling the next release 5.1. TCM and Accord are a massive thing. Other .1 / .2 releases were the .0 with some smaller things added. Imo this is a huge step forward, as big as 5.0 was, so we should call it 6.0.

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

It might be a fun experiment to retrofit the Harry tests we're currently using (once Harry 2.0 lands in trunk from cep-15-accord) to fuzz SAI and point them at legacy 2i (i.e. the subset of query types legacy 2i supports) and see if we find anything interesting, but I don't even know if that rises

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

I agree with Aleksey on how we should approach feature flags, and if we think 2i simply don’t work we should make that determination and mark them broken not deprecated. The only bug mentioned so far is 18656, which doesn’t clearly argue that the behaviour is incorrect rather than just undesire

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

I misspoke earlier about the feature gap. SAI supports queries legacy 2i does not, like numeric range queries.On Dec 10, 2024, at 9:29 AM, Caleb Rackliffe wrote:I think my point here is that the hidden table 2i implementation has known correctness/availability/operational/resource usage issues wh

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

I think my point here is that the hidden table 2i implementation has known correctness/availability/operational/resource usage issues whether it has a theoretical niche use-case or not from a query performance perspective.To Štefan’s question, yes, more or less. I’d like to at least see some succes

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

I agree with Benedict. Some of the issues with 2i should actually be significantly improved as improvements to the storage engine continue. Trie memtables and indexes should reduce the garbage overhead. Improving compaction throughout should help with the bootstrap problem. I suspect theres some a

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

> There is no reason it should ever be more capable than SAI for any > partition/token-restricted query use-case, and I don't really see how there's > any short-term path for any local 2i implementation in C* to be efficient for > anything else While I am not personally aware of much evidence p

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

Ok, so, when SAI has no feature gaps and you consider 2i to not be safe then what is preventing us from deprecating 2i as suggested? Just not enough production workloads / experience running that? Jon mentioned the performance earlier (vector search) so when that is addressed / improved then we wi

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

> I’m not convinced SAI has demonstrated a practical or theoretical capability to fully replace secondary indexes anyway. So it would be very premature to mark them deprecated. > If 2i indexes are to be marked as deprecated and SAI is beta, then what is actually the index implementation we stand b

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

There is room for a few different grades of ‘flawed’. From broken (you should really not use this anymore) to “generally usable but be careful and aware of these caveats” - with different degrees of gating/warning applied. > On 10 Dec 2024, at 11:46, Aleksey Yeshchenko wrote: > > What we’ve do

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

What we’ve done is we’ve overloaded the term ‘experimental’ to mean too many related but different ideas. We need additional, more specific terminology to disambiguate. 1. Labelling released features that were known to be unstable at release as ‘experimental’ retroactively shouldn’t happen and

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

Yes, I agree. I see it the same. On Tue, Dec 10, 2024 at 12:41 PM Benedict wrote: > I’m not convinced SAI has demonstrated a practical or theoretical > capability to fully replace secondary indexes anyway. So it would be very > premature to mark them deprecated. > > On 10 Dec 2024, at 06:29, Šte

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

I’m not convinced SAI has demonstrated a practical or theoretical capability to fully replace secondary indexes anyway. So it would be very premature to mark them deprecated.On 10 Dec 2024, at 06:29, Štefan Miklošovič wrote: ... then we should NOT mark it to be deprecated. On Tue, Dec 10, 2024 at

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

... then we should NOT mark it to be deprecated. On Tue, Dec 10, 2024 at 12:27 PM Štefan Miklošovič wrote: > I have a hard time getting used to the "terminology" here. If 2i indexes > are to be marked as deprecated and SAI is beta, then what is actually the > index implementation we stand behin

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

I have a hard time getting used to the "terminology" here. If 2i indexes are to be marked as deprecated and SAI is beta, then what is actually the index implementation we stand behind in the production? It is like we are "abandoning" the former but the latter is not bullet-proof yet. The signal it

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

> A possibility with SAI is to mark it beta while also marking 2i as > deprecated (and leaving SASI as marked). This sends a clear signal > (imho) that SAI is the recommended solution forward but also being > honest about its maturity and QA. (and leaving SASI as marked *experimental*)

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

I see value in using a beta flag in addition to an experimental flag, and that such a beta flag should see a lot more use than experimental. Java 17 definitely falls in the beta category. I/We definitely recommend its usage in production, but as has been said data is needed over trust and the co