Re: Downgradability

[Dinesh Joshi]

I’m a big fan of maintaining backward compatibility. Downgradability implies that we could potentially roll back an upgrade at any time. While I don’t think we need to retain the ability to downgrade in perpetuity it would be a good objective to maintain strict backward compatibility and therefore downgradability until a certain point. This would imply versioning metadata and extending it in such a way that prior version(s) could continue functioning. This can certainly be expensive to implement and might bloat on-disk storage. However, we could always offer an option for the operator to optimize the on-disk structures for the current version then we can rewrite them in the latest version. This optimizes the storage and opens up new functionality. This means new features that can work with old on-disk structures will be available while others that strictly require new versions of the data structures will be unavailable until the operator migrates to the new version. This migration IMO should be irreversible. Beyond this point the operator will lose the ability to downgrade which is ok. Dinesh On Feb 20, 2023, at 10:40 AM, Jake Luciani <jak...@gmail.com> wrote:  There has been progress on  https://issues.apache.org/jira/plugins/servlet/mobile#issue/CASSANDRA-8928 Which is similar to what datastax does for DSE. Would this be an acceptable solution? Jake  On Mon, Feb 20, 2023 at 11:17 AM guo Maxwell <cclive1...@gmail.com> wrote: It seems “An alternative solution is to implement/complete CASSANDRA-8110” can give us more options if it is finished😉 Branimir Lambov <blam...@apache.org>于2023年2月20日 周一下午11:03写道： Hi everyone, There has been a discussion lately about changes to the sstable format in the context of being able to abort a cluster upgrade, and the fact that changes to sstables can prevent downgraded nodes from reading any data written during their temporary operation with the new version. Most of the discussion is in CASSANDRA-18134, and is spreading into CASSANDRA-14277 and CASSANDRA-17698, none of which is a good place to discuss the topic seriously. Downgradability is a worthy goal and is listed in the current roadmap. I would like to open a discussion here on how it would be achieved. My understanding of what has been suggested so far translates to: - avoid changes to sstable formats; - if there are changes, implement them in a way that is backwards-compatible, e.g. by duplicating data, so that a new version is presented in a component or portion of a component that legacy nodes will not try to read; - if the latter is not feasible, make sure the changes are only applied if a feature flag has been enabled. To me this approach introduces several risks: - it bloats file and parsing complexity; - it discourages improvement (e.g. CASSANDRA-17698 is no longer a LHF ticket once this requirement is in place); - it needs care to avoid risky solutions to address technical issues with the format versioning (e.g. staying on n-versions for 5.0 and needing a bump for a 4.1 bugfix might require porting over support for new features); - it requires separate and uncoordinated solutions to the problem and switching mechanisms for each individual change. An alternative solution is to implement/complete CASSANDRA-8110, which provides a method of writing sstables for a target version. During upgrades, a node could be set to produce sstables corresponding to the older version, and there is a very straightforward way to implement modifications to formats like the tickets above to conform to its requirements.  What do people think should be the way forward? Regards, Branimir -- you are the apple of my eye ! -- http://twitter.com/tjake