Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

>> https://docs.google.com/spreadsheets/d/11MOxhNqwE1tWP4ex2gzKG2pmeAWFaHDKo-CRp25h9BU/edit?gid=0#gid=0 We still have a lot of rows empty. I have added many default values and a Cassandra version when a parameter was introduced (to differentiate some recent parameters from old ones) based on source

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

Good point re: the implications of parsing and durability in the face of seeing unknown or missing parameters. I don't think widening the scope on that would be ideal, especially considering the entire impetus for this conversation is "we've misbehaved with our config and have a bunch of undocum

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

"we take "unclaimed" items and move them to their own InternalConfig.java or something" This is interesting. If we are meant to be still able to put these properties into cassandra.yaml (even they are "internal ones") and they would be just in InternalConfig.java for some basic separation of inter

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

This may be an off-base comparison, but this reminds me of struggles we've had getting to 0 failing unit tests before and the debates on fencing off a snapshot of the current "failure set" so you can have a set point where no further degradation is allowed in a primary data set. All of which is

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

Indeed, we need to balance that and thoughtfully choose what is going to be added and what not. However, we should not hide something which is meant to be tweaked by a user. The config is intimidating mostly because everything is just in one file. I merely remember discussions a few years ago which

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

Might be a bit of a balance between exposing what people actually are likely to need to modify vs having a super intimidating config file. It's already nearly 2000 lines. Personally I'd rather see some auto-documentation or something that's in the docs

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

I have compiled this table (1). It is editable by everybody. The idea behind that is that if people recognize they would like to have some property in cassandra.yaml exposed, just mark it as green, add a comment and some notes. It would be cool if we did this together as people who have introduced

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

> I think we need to integrate this to some ant target. If you expanded on this, that would be great. A draft version of the ant target (as of now it is configured as non-failed and not attached to the usual build process): https://github.com/apache/cassandra/pull/3830/files On Fri, 24 Jan 2025

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

How are we going to document what each of 112 missing properties is doing and / or exclude them from cassandra.yaml? There are a lot of properties which just don't ring a bell exactly what they are for. I think we should create a basic table and document what each is for and what is the decision ab

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

So if you forget to update cassandra.yaml you fail the build? Makes sense to me. One additional thing I would like to see is the reverse… did you put something in yaml that isn’t in Config? This is a bug I have seen a few times…. Mostly because people don’t know the rules from SnakeYAML so ar

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

Love the suggestion of marking the hidden/advanced configuration properties with annotations. Leaving a “configuration” property out of the main configuration file should be deliberate and well thought and argued. I highly doubt we have 112 “advanced” properties that really need to be hidden to

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

Love the suggestion of marking the hidden/advanced configuration properties with annotations. Leaving a “configuration” property out of the main configuration file should be deliberate and well thought and argued. I highly doubt we have 112 “advanced” properties that really need to be hidden to

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

https://issues.apache.org/jira/browse/CASSANDRA-20249 On Fri, 24 Jan 2025 at 15:40, Dmitry Konstantinov wrote: > Maybe I missed some patterns but it looks like a pretty good estimation, I > did like 10 random checks manually to verify :-) > I will try to make an ant target with a similar logic (

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

Maybe I missed some patterns but it looks like a pretty good estimation, I did like 10 random checks manually to verify :-) I will try to make an ant target with a similar logic (hopefully, during the weekend) I will create a ticket to track this activity (to share attachments there to not overload

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

Oh my god, 112? :DD I was thinking it would be less than 10. Anyway, I think we need to integrate this to some ant target. If you expanded on this, that would be great. On Fri, Jan 24, 2025 at 4:31 PM Dmitry Konstantinov wrote: > A very primitive implementation of the 1st idea below: > > String

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

A very primitive implementation of the 1st idea below: String configUrl = "file:///Users/dmitry/IdeaProjects/cassandra-trunk/conf/cassandra.yaml"; Field[] allFields = Config.class.getFields(); List topLevelPropertyNames = new ArrayList<>(); for(Field field : allFields) { if (!Modifier.isStatic

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

It should also work the other way around. If there is a property which is commented out in yaml and it is not in Config.java, that should fail as well. If it is not commented out and it is not in Config.java, that will fail in runtime as it fails on unrecognized property. This will be used in prac

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

> > If "# my_cool_property: true" is NOT in cassandra.yaml, we might indeed add it, also commented out. I think it would be quite easy to check against yaml if there is a line starting on "# my_cool_property" or just on "my_cool_property". Both cases would satisfy the check. Makes sense, I think

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

+1 to making hidden properties explicit. No way to tell whether it's a mistake or not atm. On Fri, Jan 24, 2025, at 9:36 AM, Štefan Miklošovič wrote: > > > On Fri, Jan 24, 2025 at 3:27 PM Paulo Motta wrote: >> > from time to time I see configuration properties in Config.java and they >> > are

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

On Fri, Jan 24, 2025 at 3:27 PM Paulo Motta wrote: > > from time to time I see configuration properties in Config.java and they > are clearly not in cassandra.yaml. Not every property in Config is in > cassandra.yaml. I would like to know if there is some specific reason > behind that. > > I thin

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

Same understanding here - the goal of property being presented in Config but not cassandra.yaml was to keep those hidden from regular users aka - advanced config. Also, in some cases I was told some of those did not have a clear recommendation yet how to use them, one more reason to be hidden. I t

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

Hi, I've just thought about the same as Stefan - can we automate this consistency check.. (the topic is actually triggered by our recent review). I definitely saw some other existing options which are available in Config but not mentioned in cassandra.yaml. The case with intentionally hidden option

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

> from time to time I see configuration properties in Config.java and they are clearly not in cassandra.yaml. Not every property in Config is in cassandra.yaml. I would like to know if there is some specific reason behind that. I think one of the original reasons was to "hide" advanced configs tha

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

I wouldn't oppose making things more well defined there. Kind Regards, Brandon On Fri, Jan 24, 2025 at 8:22 AM Štefan Miklošovič wrote: > > If that is the case, we could have something like this? > > @HiddenInYaml > public boolean auto_bootstrap = true; > > Then a build-time check would

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

If that is the case, we could have something like this? @HiddenInYaml public boolean auto_bootstrap = true; Then a build-time check would parse Config.java on these annotations etc and skip that if it is not in yaml. It is OK if we decide not to expose but we should have some basic overv

Re: [DISCUSS] synchronisation of properties between Config.java and cassandra.yaml

I think those are things we've decided we don't want to expose to the average user, like auto_bootstrap. Kind Regards, Brandon On Fri, Jan 24, 2025 at 8:16 AM Štefan Miklošovič wrote: > > Hello, > > from time to time I see configuration properties in Config.java and they are > clearly not in ca