Re: [DISCUSS] Nested YAML configs for new features

2021-11-29 Thread David Capwell
> but I would hate to repeat the mistakes of our past by evolving the config > in a new direction without any coherent overarching design. At the start I asked to keep the thread local to new features, but to more flesh out an “overarching design” maybe we should increase the “desired” scope t

Re: [DISCUSS] Throughput issues when inserting on contended partitions

2021-11-29 Thread bened...@apache.org
I’m in favour, though I have weaker requirements for backports than others. This work is pretty significant, though. It’s nothing like the complexity of CEP-14, but it heavily modifies a critical piece of the system. I would say that it needs a rigorous review process if it’s going into a patch

Re: [DISCUSS] Nested YAML configs for new features

2021-11-29 Thread bened...@apache.org
If we’re thinking of moving towards nested configuration, then before employing the approach further we would ideally consider what a fully nested config looks like for the project. Ekaterina has done a lot to clean up inconsistent naming, but I would hate to repeat the mistakes of our past by e

Re: [DISCUSS] Nested YAML configs for new features

2021-11-29 Thread David Capwell
> What should our default example cassandra.yaml file use (flat or nested)? > Currently default shows nested Was told this statement was confusing, so trying to clarify. At the moment we do not allow a nested config to be expressed in any way outside of nesting it (excluding YAML’s ability to

Re: [DISCUSS] Nested YAML configs for new features

2021-11-29 Thread David Capwell
Thanks everyone for the comments, I hope below is a good summary of all the talking points? We already use nested configs (networking, seed provider, commit log/hint compression, back pressure, etc.) Flat configs are easier for grep, but can be solved with grep -A/-B and/or yq It would be possib

Cassandra project biweekly status update 2021-11-29

2021-11-29 Thread Joshua McKenzie
Sorry for the miss last week; it being a holiday in the US meant I was on the road managing tiny humans and a puppy with my partner and I failed to hand off update email responsibility to someone else. Which means we have three weeks to cover! [New contributor Getting Started] As a new contributor

Re: [DISCUSS] Nested YAML configs for new features

2021-11-29 Thread bened...@apache.org
I meant to imply we should improve our UDT usability to support this kind of querying, essentially – but that if we support a simple text->property setup we might want to offer LIKE support so we can search them (via simple filtering, not any index) – which is actually pretty easy to provide. I

Re: [RESULT] [VOTE] CEP-10: Cluster and Code Simulations

2021-11-29 Thread bened...@apache.org
FYI, CASSANDRA-17008 (the main element of CEP-10) is ready to merge, in case anybody still plans to take a look. Otherwise it will land in a day or two. From: bened...@apache.org Date: Friday, 30 July 2021 at 14:27 To: dev@cassandra.apache.org Subject: [RESULT] [VOTE] CEP-10: Cluster and Code S

Re: [DISCUSS] Nested YAML configs for new features

2021-11-29 Thread Joseph Lynch
On Mon, Nov 29, 2021 at 11:51 AM bened...@apache.org wrote: > > Maybe we can make our query language more expressive 😊 > > We might anyway want to introduce e.g. a LIKE filtering option to > find/discover flattened config parameters? This sounds more complicated than just having the settings vir

Re: [DISCUSS] Throughput issues when inserting on contended partitions

2021-11-29 Thread Brandon Williams
I think it makes a lot of sense to fix these in 4.0, they have been lingering issues long enough. +1 On Mon, Nov 29, 2021 at 10:59 AM Benjamin Lerer wrote: > > Hi everybody, > > We have seen some serious throughput issues when inserting data with > collections on contended partitions ( CASSANDRA-

Re: [DISCUSS] Nested YAML configs for new features

2021-11-29 Thread Benjamin Lerer
> > We might anyway want to introduce e.g. a LIKE filtering option to > find/discover flattened config parameters? +100 Le lun. 29 nov. 2021 à 17:51, bened...@apache.org a écrit : > Maybe we can make our query language more expressive 😊 > > We might anyway want to introduce e.g. a LIKE filteri

[DISCUSS] Throughput issues when inserting on contended partitions

2021-11-29 Thread Benjamin Lerer
Hi everybody, We have seen some serious throughput issues when inserting data with collections on contended partitions ( CASSANDRA-17163 and CASSANDRA-15464 ). Benedict has created some p

Re: [DISCUSS] Nested YAML configs for new features

2021-11-29 Thread bened...@apache.org
Maybe we can make our query language more expressive 😊 We might anyway want to introduce e.g. a LIKE filtering option to find/discover flattened config parameters? From: Benjamin Lerer Date: Monday, 29 November 2021 at 16:41 To: dev@cassandra.apache.org Subject: Re: [DISCUSS] Nested YAML confi

Re: [DISCUSS] Nested YAML configs for new features

2021-11-29 Thread Benjamin Lerer
> > I don’t think it’s necessarily a requirement that we use the flattened > version in vtables. At the very least we can make use of sets, lists, etc. > But we can probably also use UDTs if this improves clarity. In my opinion part of the issue is on the query side. How do we select a nested set

Re: [DISCUSS] Nested YAML configs for new features

2021-11-29 Thread Bowen Song
In ElasticSearch, the default is a flattened format with almost all lines commented out. See https://github.com/elastic/elasticsearch/blob/master/distribution/src/config/elasticsearch.yml I guess they chose to do that because user can uncomment individual lines to make changes. In a structured

Re: [DISCUSS] Nested YAML configs for new features

2021-11-29 Thread bened...@apache.org
I don’t think it’s necessarily a requirement that we use the flattened version in vtables. At the very least we can make use of sets, lists, etc. But we can probably also use UDTs if this improves clarity. From: Benjamin Lerer Date: Monday, 29 November 2021 at 15:54 To: dev@cassandra.apache.org

Re: [DISCUSS] Nested YAML configs for new features

2021-11-29 Thread Benjamin Lerer
I do not think that supporting both options is an issue. The settings virtual table would have to use the flattened version. If we support both formats, the question would be: what should be the one used by default in the configuration file? Le ven. 26 nov. 2021 à 15:40, bened...@apache.org a écr