So the weakest version of the plan that actually accomplishes something useful for 5.0:
1.) Just leave the CREATE INDEX default alone for now. Hard switch the default after 5.0. 2.) Add USING...WITH... support to CREATE INDEX, so we don't have to go to market using CREATE CUSTOM INDEX, which feels...not so polished. (The backend for this already exists w/ CREATE CUSTOM INDEX.) 3.) Leave in place but deprecate (client warnings could work?) CREATE CUSTOM INDEX. Support the syntax for the foreseeable future. Any objections to that? On Fri, May 12, 2023 at 12:10 PM Caleb Rackliffe <calebrackli...@gmail.com> wrote: > I don't want to cut over for 5.0 either way. I was more contrasting a > configurable cutover in 5.0 vs. a hard cutover later. > > On Fri, May 12, 2023 at 12:09 PM Benedict <bened...@apache.org> wrote: > >> If the performance characteristics are as clear cut as you think, then >> maybe it will be an easy decision once the evidence is available for >> everyone to consider? >> >> If not, then we probably can’t do the hard cutover and so the answer is >> still pretty simple? >> >> On 12 May 2023, at 18:04, Caleb Rackliffe <calebrackli...@gmail.com> >> wrote: >> >> >> I don't particularly like the YAML solution either, but absent that, >> we're back to fighting about whether we introduce entirely new syntax or >> hard cut over to SAI at some point. >> >> We already have per-node configuration in the YAML that determines >> whether or not we can create a 2i at all, right? >> >> What if we just do #2 and #3 and punt on everything else? >> >> On Fri, May 12, 2023 at 11:56 AM Benedict <bened...@apache.org> wrote: >> >>> A table is not a local concept at all, it has a global primary index - >>> that’s the core idea of Cassandra. >>> >>> I agree with Brandon that changing CQL behaviour like this based on node >>> config is really not ideal. New syntax is by far the simplest and safest >>> solution to this IMO. It doesn’t have to use the word LOCAL, but I think >>> that’s anyway an improvement, personally. >>> >>> In future we will hopefully offer GLOBAL indexes, and IMO it is better >>> to reify the distinction in the syntax. >>> >>> On 12 May 2023, at 17:29, Caleb Rackliffe <calebrackli...@gmail.com> >>> wrote: >>> >>> >>> We don't need to know everything about SAI's performance profile to plan >>> and execute some small, reasonable things now for 5.0. I'm going to try to >>> summarize the least controversial package of ideas from the discussion >>> above. I've left out creating any new syntax. For example, I think CREATE >>> LOCAL INDEX, while explicit, is just not necessary. We don't use CREATE >>> LOCAL TABLE, although it has the same locality as our indexes. >>> >>> Okay, so the proposal for 5.0... >>> >>> 1.) Add a YAML option that specifies a default implementation for CREATE >>> INDEX, and make this the legacy 2i for now. No existing DDL breaks. We >>> don't have to commit to the absolute superiority of SAI. >>> 2.) Add USING...WITH... support to CREATE INDEX, so we don't have to go >>> to market using CREATE CUSTOM INDEX, which feels...not so polished. >>> (The backend for this already exists w/ CREATE CUSTOM INDEX.) >>> 3.) Leave in place but deprecate (client warnings could work?) CREATE >>> CUSTOM INDEX. Support the syntax for the foreseeable future. >>> >>> Can we live w/ this? >>> >>> I don't think any information about SAI we could possibly acquire before >>> a 5.0 release would affect the reasonableness of this much. >>> >>> >>> On Fri, May 12, 2023 at 10:54 AM Benedict <bened...@apache.org> wrote: >>> >>>> if we didn't have copious amounts of (not all public, I know, working >>>> on it) evidence >>>> >>>> >>>> If that’s the assumption on which this proposal is based, let’s discuss >>>> the evidence base first, as given the fundamentally different way they work >>>> (almost diametrically opposite), I would want to see a very high quality of >>>> evidence to support the claim. >>>> >>>> I don’t think we can resolve this conversation effectively until this >>>> question is settled. >>>> >>>> On 12 May 2023, at 16:19, Caleb Rackliffe <calebrackli...@gmail.com> >>>> wrote: >>>> >>>> >>>> > This creates huge headaches for everyone successfully using 2i today >>>> though, and SAI *is not* guaranteed to perform as well or better - it has a >>>> very different performance profile. >>>> >>>> We wouldn't have even advanced it to this point if we didn't have >>>> copious amounts of (not all public, I know, working on it) evidence it did >>>> for the vast majority of workloads. Having said that, I don't strongly >>>> agree that we should make it the default in 5.0, because performance isn't >>>> the only concern. (correctness, DDL back-compat, which we've sort of >>>> touched w/ the YAML default option, etc.) >>>> >>>> This conversation is now going in like 3 different directions, or at >>>> least 3 different "packages" of ideas, so there isn't even a single thing >>>> to vote on. Let me read through again and try to distill into something >>>> that we might be able to do so with... >>>> >>>> On Fri, May 12, 2023 at 7:56 AM Aleksey Yeshchenko <alek...@apple.com> >>>> wrote: >>>> >>>>> This. >>>>> >>>>> I would also consider adding CREATE LEGACY INDEX syntax as an alias >>>>> for today’s CREATE INDEX, the latter to be deprecated and (in very distant >>>>> future) removed. >>>>> >>>>> On 12 May 2023, at 13:14, Benedict <bened...@apache.org> wrote: >>>>> >>>>> This creates huge headaches for everyone successfully using 2i today >>>>> though, and SAI *is not* guaranteed to perform as well or better - it has >>>>> a >>>>> very different performance profile. >>>>> >>>>> I think we should deprecate CREATE INDEX, and introduce new syntax >>>>> CREATE LOCAL INDEX to make clear that this is not a global index, and that >>>>> this should require the USING syntax to avoid this problem in future. >>>>> >>>>> We should report warnings to the client when CREATE INDEX is used, >>>>> indicating it is deprecated. >>>>> >>>>> >>>>>
