Re: Implicitly enabling ALLOW FILTERING on virtual tables

2023-02-03 Thread Miklosovic, Stefan
This is the draft for FILTERING ON|OFF in shell. I would say this is the most simple solution. We may still consider table option but what do you think about having it simply just set via shell? https://github.com/apache/cassandra/pull/2141/files From:

Re: Implicitly enabling ALLOW FILTERING on virtual tables

2023-02-03 Thread Josh McKenzie
> they would start to set ALLOW FILTERING here and there in order to not think > twice about their data model so they can just call it a day. Setting this on a per-table basis or having users set this on specific queries that hit tables and forgetting they set it are 6 of one and half-a-dozen of

Re: Important news about Cassandra Summit

2023-02-03 Thread Wei Deng
Thank you for the update, Patrick. Appreciate all the work your team, the community and the Linux Foundation Events team have done. Looking forward to the virtual event and Cassandra Day series. > If you have open positions, call them out in this email thread or #cassandra in the ASF slack. Speak

Re: Implicitly enabling ALLOW FILTERING on virtual tables

2023-02-03 Thread Miklosovic, Stefan
Yes, there would be discrepancy. I do not like that either. If it was only about "normal tables vs virtual tables", I could live with that. But the fact that there are going to be differences among vtables themselves, that starts to be a little bit messy. Then we would need to let operators know

Re: Implicitly enabling ALLOW FILTERING on virtual tables

2023-02-03 Thread David Capwell
> I don't think the assumption that "virtual tables will always be small and > always fit in memory" is a safe one. Agree, there is a repair ticket to have the coordinating node do network queries to peers to resolve the table (rather than operator querying everything, allow the coordinator nod

Important news about Cassandra Summit

2023-02-03 Thread Patrick McFadin
*Hello Cassandra Community,We all see what’s happening in tech right now. Cuts are being made, and budgets are frozen. For Cassandra Summit, this has translated to low sponsorship and registrations. The program committee has been discussing options with the Linux Foundation events team, and the dec

Re: Implicitly enabling ALLOW FILTERING on virtual tables

2023-02-03 Thread Miklosovic, Stefan
While that might technically work, Benedict, I am afraid that if we enable users to have this kind of power, they would start to set ALLOW FILTERING here and there in order to not think twice about their data model so they can just call it a day. At the same time, we have a guardrail for allowi

Re: Implicitly enabling ALLOW FILTERING on virtual tables

2023-02-03 Thread Benedict
Why not introduce a general table option that toggles ALLOW FILTERING behaviour and just flip it for virtual tables we want this behaviour for? Users can do it too, for their own tables for which it’s suitable.On 3 Feb 2023, at 20:59, Andrés de la Peña wrote:For those eventual big virtual tables

Re: Implicitly enabling ALLOW FILTERING on virtual tables

2023-02-03 Thread Chris Lohfink
Yes, I am not -1. Just that if we do it we should be ok in the future with some virtual tables that did not have this behavior. Should consider if this would be confusing. Really should be ok imho since they just would get the "need allow filtering" error on said future tables. Chris On Fri, Feb

Re: Implicitly enabling ALLOW FILTERING on virtual tables

2023-02-03 Thread Andrés de la Peña
For those eventual big virtual tables there is the mentioned flag indicating whether the table allows filtering without AF. I guess the question is how can a user know whether a certain virtual table is one of the big ones. That could be specified in the doc for each table, and it could also be in

Re: Implicitly enabling ALLOW FILTERING on virtual tables

2023-02-03 Thread Chris Lohfink
Just to 2nd what Scott days. While everything is in memory now, it may not be in the future, and if we add it implicitly, we are tying ourselves to be in memory only. However, I wouldn't -1 the idea. Another option may be a cqlsh option (ie like expand on/off) to always include a flag so it doesnt

Re: Welcome Patrick McFadin as Cassandra Committer

2023-02-03 Thread C. Scott Andreas
Congratulations, Patrick!On Feb 2, 2023, at 9:46 PM, Berenguer Blasi wrote:Welcome!On 3/2/23 4:09, Vinay Chella wrote:Well deserved one, Congratulations, Patrick. On Fri, Feb 3, 2023 at 4:01 AM Josh McKenzie wrote:Congrats Patrick! Well deserved.On Thu, Feb 2, 2023, at 5:

Re: Implicitly enabling ALLOW FILTERING on virtual tables

2023-02-03 Thread C. Scott Andreas
There are some ideas that development community members have kicked around that may falsify the assumption that "virtual tables are tiny and will fit in memory."One example is CASSANDRA-14629: Abstract Virtual Table for very large result setshttps://issues.apache.org/jira/browse/CASSANDRA-14629C

Re: [DISCUSS] Merging incremental feature work

2023-02-03 Thread Derek Chen-Becker
I think Henrik has a lot of good points. However, I want to point out that JDK upgrades are non-optional over the fullness of time, so it might be worth carving out a specific process for that work. In a similar vein, security patches also Just Need to Merge™, so I'm a little hesitant when I see th

Re: [DISCUSS] Merging incremental feature work

2023-02-03 Thread Ekaterina Dimitrova
I am all in for incremental changes which are fine to get into release. In the case of JDK 17 I know dependencies which need to be updated for Java 17 but they still work with Java8 and 11 so it is fine to update them before the switch. So while some blockers are removed, I do the updates in parall

Re: Implicitly enabling ALLOW FILTERING on virtual tables

2023-02-03 Thread Maxim Muzafarov
Hello Stefan, Regarding the decision to implicitly enable ALLOW FILTERING for virtual tables, which also makes sense to me, it may be necessary to consider changing the clustering columns in the virtual table metadata to regular columns as well. The reasons are the same as mentioned earlier: the v

Re: [DISCUSS] Merging incremental feature work

2023-02-03 Thread Josh McKenzie
> *Suddenly, you get to be doubly managerial for making _two_ inconsequential > decisions - the wrong one _and_ the right one.* Something to aspire to. :) On Fri, Feb 3, 2023, at 9:20 AM, Henrik Ingo wrote: > I've been an unusually active debater recently, so it might be appropriate to > start

Re: CASSANDRA-14227 removing the 2038 limit

2023-02-03 Thread Henrik Ingo
In that case I agree that increasing from 20 years is an interesting opportunity but clearly out of scope for your current ticket. On Fri, Feb 3, 2023 at 3:48 PM Berenguer Blasi wrote: > Hi, > > 20y is the current and historic value. 68y is what an integer can > accommodate hence the current 203

Re: [DISCUSS] Merging incremental feature work

2023-02-03 Thread Henrik Ingo
I've been an unusually active debater recently, so it might be appropriate to start with a reminder/disclaimer that I'm not actually a Cassandra contributor in any way(*), but I choose to share some thoughts where I feel that sharing experiences from other open source database projects can be of us

Re: [DISCUSS] API modifications and when to raise a thread on the dev ML

2023-02-03 Thread Berenguer Blasi
Hi All, I don't think we can be 100% prescriptive here. There will always be gray areas on where API changes start or end. Even on what is an API change and discussions about why a change was forgotten, not communicated, etc. I think we should rely on people's judgement on when to hit the ML/

Re: CASSANDRA-14227 removing the 2038 limit

2023-02-03 Thread Berenguer Blasi
Hi, 20y is the current and historic value. 68y is what an integer can accommodate hence the current 2038 limit since the 1970 Unix epoch. I wouldn't make it a configurable value, off the top of my head it would make for some interesting bugs and debugging sessions when nodes had different val

Re: [DISCUSS] Merging incremental feature work

2023-02-03 Thread Andrés de la Peña
Yes, I think that some refactors can also be directly merged if they have a value for the end user on their own. Changes providing cleaner, better documented and less tightly coupled code can have that value, even if they aren't a new feature. Things like new APIs without an implementation probably

Re: CASSANDRA-14227 removing the 2038 limit

2023-02-03 Thread Henrik Ingo
Naive PHB questions to follow... Why are 68y and 20y special? Could you pick any value? Could we allow it to be configurable? (Last one probably overkill, just asking to understand...) If we can pick any values we want, instinctively I would personally suggest to have TTL higher than 20 years, bu

Re: [DISCUSS] Merging incremental feature work

2023-02-03 Thread Josh McKenzie
The deeply coupled nature of some areas of our codebase does have some constraints it imposes on us here to your point. Without sensible internal APIs a lot of this type of work expands into two phases, one to refactor out said APIs and the other to introduce new functionality. It probably dep

Re: [DISCUSS] Merging incremental feature work

2023-02-03 Thread Andrés de la Peña
I'm not sure a CEP is necessarily a big, complex piece of work that needs to be split into multiple tickets. There could be single-ticket CEPs that don't need multiple tickets, and still need the bureaucracy of a CEP due to the impact of the change. However that probably isn't the common case. Als

Re: [DISCUSS] Merging incremental feature work

2023-02-03 Thread Sam Tunnicliffe
This is quite timely as we're just gearing up to begin pushing the work we've been doing on CEP-21 into the public domain. This CEP is a slightly different from others that have gone before in that it touches almost every area of the system. This presents a few implementation challenges, most

Re: [DISCUSS] Merging incremental feature work

2023-02-03 Thread Josh McKenzie
Anything we either a) have to do (JDK support) or b) have all agreed up front we think we should do (CEP). I.e. things with a lower risk of being left dead in the codebase partially implemented. I don't think it's a coincidence we've set up other processes to help de-risk and streamline the con

Re: [DISCUSS] Merging incremental feature work

2023-02-03 Thread Brandon Williams
On Fri, Feb 3, 2023 at 6:06 AM Josh McKenzie wrote: > > My current thinking: I'd like to propose we all agree to move to merge work > into trunk incrementally if it's either: > 1) New JDK support > 2) An approved CEP So basically everything? I'm not sure what large complex bodies of work would

[DISCUSS] Merging incremental feature work

2023-02-03 Thread Josh McKenzie
The topic of how we handle merging large complex bodies of work came up recently with the CEP-15 merge and JDK17, and we've faced this question in the past as well (CASSANDRA-8099 comes to mind). The times we've done large bodies of work separately from trunk and then merged them in have their

Re: Implicitly enabling ALLOW FILTERING on virtual tables

2023-02-03 Thread Andrés de la Peña
I think removing the need for ALLOW FILTERING on virtual tables makes sense and would be quite useful for operators. That guard exists for performance issues that shouldn't occur on virtual tables. We also have a flag in case some future virtual table implementation has limitations regarding filte

Re: Introducing mockito-inline library among test dependencies

2023-02-03 Thread Miklosovic, Stefan
Hi list, I forgot to update this thread. I just want to let you know that since we merged CASSANDRA-14361 to trunk which introduced mockito-inline dependency, you can mock static methods in trunk already. Regards From: Miklosovic, Stefan Sent: Thursday

Re: CASSANDRA-14227 removing the 2038 limit

2023-02-03 Thread Berenguer Blasi
Hi All, a version using Uints, 20y max TTL and kicking the can down the road until 2086 has been put up for review #justfyi Regards On 15/11/22 7:06, Berenguer Blasi wrote: Hi all, thanks for your answers!. To Benedict's point: In terms of the uvint enconding of deletionTime i.e. it is t

Implicitly enabling ALLOW FILTERING on virtual tables

2023-02-03 Thread Miklosovic, Stefan
Hi list, the content of virtual tables is held in memory (and / or is fetched every time upon request). While doing queries against such table for a column outside of primary key, normally, users are required to specify ALLOW FILTERING. This makes total sense for "ordinary tables" for applicati