Paulo: > 1) Will cassandra.yaml remain the default test config? Is the plan moving > forward to require green CI for both configurations on pre-commit, or > pre-release?
The plan is to ensure both configurations are green pre-commit. This should not increase the CI cost as this replaces extra configurations we were running before (e.g. test-tries). 2) What will this mean for the release artifact, is the idea to continue > shipping with the current cassandra.yaml or eventually switch to the > optimized configuration (ie. 6.X) while making the legacy default > configuration available via an optional flag? The release simply includes an additional yaml file, which contains a one-liner how to use it. Jeff: > 1) If there’s an “old compatible default” and “latest recommended > settings”, when does the value in “old compatible default” get updated? > Never? This does not change anything about these decisions. The question is very serious without this patch as well: Does V6 have to support pain-free upgrade from V5 working in V4 compatible mode? If so, can we ever deprecate or drop anything? If not, are we not breaking upgradeability promises? 2) If there are test failures with the new values, it seems REALLY > IMPORTANT to make sure those test failures are discovered + fixed IN THE > FUTURE TOO. If pushing new yaml into a different file makes us less likely > to catch the failures in the future, it seems like we’re hurting ourselves. > Branimir mentions this, but how do we ensure that we don’t let this pattern > disguise future bugs? The main objective of this patch is to ensure that the second yaml is tested too, pre-commit. We were not doing this for all features we tell users are supported. Paulo: > - if cassandra_latest.yaml becomes the new default configuration for 6.0, > then precommit only needs to be run against thatversion - prerelease needs > to be run against all cassandra.yaml variants. Assuming we keep the pace of development, there will be new "latest" features in 6.0 (e.g. Accord could be one). The idea is more to move some of the settings from latest to default when they are deemed mature enough. Josh: > I propose to significantly reduce that stuff. Let's distinguish the > packages of tests that need to be run with CDC enabled / disabled, with > commitlog compression enabled / disabled, tests that verify sstable formats > (mostly io and index I guess), and leave other parameters set as with the > latest configuration - this is the easiest way I think. > For dtests we have vnodes/no-vnodes, offheap/onheap, and nothing about > other stuff. To me running no-vnodes makes no sense because no-vnodes is > just a special case of vnodes=1. On the other hand offheap/onheap buffers > could be tested in unit tests. In short, I'd run dtests only with the > default and latest configuration. Some of these changes are already done in this ticket. Regards, Branimir On Thu, Feb 15, 2024 at 3:08 PM Paulo Motta <pa...@apache.org> wrote: > > It's also been questioned about why we don't just enable settings we > recommend. These are settings we recommend for new clusters. *Our > existing cassandra.yaml needs to be tailored for existing clusters being > upgraded, where we are very conservative about changing defaults.* > > I think this unnecessarily penalizes new users with subpar defaults and > existing users who wish to use optimized/recommended defaults and need to > maintain additional logic to support that. This change offers an > opportunity to revisit this. > > Is not updating the default cassandra.yaml with new recommended > configuration just to protect existing clusters from accidentally > overriding cassandra.yaml with a new version during major upgrades? If so, > perhaps we could add a new explicit flag “enable_major_upgrade: false” to > “cassandra.yaml” that fails startup if an upgrade is detected and force > operators to review the configuration before a major upgrade? > > Related to Jeff’s question, I think we need a way to consolidate “latest > recommended settings” into “old compatible default” when cutting a new > major version, otherwise the files will diverge perpetually. > > I think cassandra_latest.yaml offers a way to “buffer” proposals for > default configuration changes which are consolidated into “cassandra.yaml” > in the subsequent major release, eventually converging configurations and > reducing the maintenance burden. > > On Thu, 15 Feb 2024 at 04:24 Mick Semb Wever <m...@apache.org> wrote: > >> >> >>> Mick and Ekaterina (and everyone really) - any thoughts on what test >>> coverage, if any, we should commit to for this new configuration? >>> Acknowledging that we already have *a lot* of CI that we run. >>> >> >> >> >> Branimir in this patch has already done some basic cleanup of test >> variations, so this is not a duplication of the pipeline. It's a >> significant improvement. >> >> I'm ok with cassandra_latest being committed and added to the pipeline, >> *if* the authors genuinely believe there's significant time and effort >> saved in doing so. >> >> How many broken tests are we talking about ? >> Are they consistently broken or flaky ? >> Are they ticketed up and 5.0-rc blockers ? >> >> Having to deal with flakies and broken tests is an unfortunate reality to >> having a pipeline of 170k tests. >> >> Despite real frustrations I don't believe the broken windows analogy is >> appropriate here – it's more of a leave the campground cleaner… That >> being said, knowingly introducing a few broken tests is not that either, >> but still having to deal with a handful of consistently breaking tests >> for a short period of time is not the same cognitive burden as flakies. >> There are currently other broken tests in 5.0: VectorUpdateDeleteTest, >> upgrade_through_versions_test; are these compounding to the frustrations ? >> >> It's also been questioned about why we don't just enable settings we >> recommend. These are settings we recommend for new clusters. Our existing >> cassandra.yaml needs to be tailored for existing clusters being upgraded, >> where we are very conservative about changing defaults. >> >> -- Branimir Lambov e. branimir.lam...@datastax.com w. www.datastax.com
