> > It seems to me the real issue is nobody noticed this was agreed and/or > forgot and didn’t think about it much.
I have some trouble keeping up with all the discussions those days so I might have missed the discussion. The only place I remembered where this subject was mentioned was during the roadmap discussion over a year ago. My understanding of that discussion was only that there was a wish to build a solution for that problem. Are you referring to another discussion I missed? Le mar. 21 févr. 2023 à 08:41, Jacek Lewandowski < lewandowski.ja...@gmail.com> a écrit : > I'd like to mention CASSANDRA-17056 (CEP-17) here as it aims to introduce > multiple sstable formats support. It allows for providing an implementation > of SSTableFormat along with SSTableReader and SSTableWriter. That could be > extended easily to support different implementations for certain version > ranges, like one impl for ma-nz, other for oa+, etc. without having a > confusing implementation with a lot of conditional blocks. Old formats in > such case could be maintained separately from the main code and easily > switched any time. > > thanks > - - -- --- ----- -------- ------------- > Jacek Lewandowski > > > wt., 21 lut 2023 o 01:46 Yuki Morishita <yu...@apache.org> napisał(a): > >> Hi, >> >> What I wanted to address in my comment in CASSANDRA-8110( >> https://issues.apache.org/jira/browse/CASSANDRA-8110?focusedCommentId=17641705&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17641705) >> is to focus on better upgrade experience. >> >> Upgrading the cluster can be painful for some orgs with mission critical >> Cassandra cluster, where they cannot tolerate less availability because of >> the inability to replace the downed node. >> They also need to plan rolling back to the previous state when something >> happens along the way. >> The change I proposed in CASSANDRA-8110 is to achieve the goal of at >> least enabling SSTable streaming during the upgrade by not upgrading the >> SSTable version. This can make the cluster to easily rollback to the >> previous version. >> Downgrading SSTable is not the primary focus (though Cassandra needs to >> implement the way to write SSTable in older versions, so it is somewhat >> related.) >> >> I'm preparing the design doc for the change. >> Also, if I should create a separate ticket from CASSANDRA-8110 for the >> clarity of the goal of the change, please let me know. >> >> >> On Tue, Feb 21, 2023 at 5:31 AM Benedict <bened...@apache.org> wrote: >> >>> FWIW I think 8110 is the right approach, even if it isn’t a panacea. We >>> will have to eventually also tackle system schema changes (probably not >>> hard), and may have to think a little carefully about other things, eg with >>> TTLs the format change is only the contract about what values can be >>> present, so we have to make sure the data validity checks are consistent >>> with the format we write. It isn’t as simple as writing an earlier version >>> in this case (unless we permit truncating the TTL, perhaps) >>> >>> On 20 Feb 2023, at 20:24, Benedict <bened...@apache.org> wrote: >>> >>> >>> >>> In a self-organising community, things that aren’t self-policed >>> naturally end up policed in an adhoc manner, and with difficulty. I’m not >>> sure that’s the same as arbitrary enforcement. It seems to me the real >>> issue is nobody noticed this was agreed and/or forgot and didn’t think >>> about it much. >>> >>> But, even without any prior agreement, it’s perfectly reasonable to >>> request that things do not break compatibility if they do not need to, as >>> part of the normal patch integration process. >>> >>> Issues with 3.1->4.0 aren’t particularly relevant as they predate any >>> agreement to do this. But we can and should address the problem of new >>> columns in schema tables, as this happens often between versions. I’m not >>> sure it has in 4.1 though? >>> >>> Regarding downgrade versions, surely this should simply be the same as >>> upgrade versions we support? >>> >>> >>> On 20 Feb 2023, at 20:02, Jeff Jirsa <jji...@gmail.com> wrote: >>> >>> >>> I'm not even convinced even 8110 addresses this - just writing sstables >>> in old versions won't help if we ever add things like new types or new >>> types of collections without other control abilities. Claude's other email >>> in another thread a few hours ago talks about some of these surprises - >>> "Specifically during the 3.1 -> 4.0 changes a column broadcast_port was >>> added to system/local. This means that 3.1 system can not read the table >>> as it has no definition for it. I tried marking the column for deletion in >>> the metadata and in the serialization header. The later got past the >>> column not found problem, but I suspect that it just means that data >>> columns after broadcast_port shifted and so incorrectly read." - this is a >>> harder problem to solve than just versioning sstables and network >>> protocols. >>> >>> Stepping back a bit, we have downgrade ability listed as a goal, but >>> it's not (as far as I can tell) universally enforced, nor is it clear at >>> which point we will be able to concretely say "this release can be >>> downgraded to X". Until we actually define and agree that this is a >>> real goal with a concrete version where downgrade-ability becomes real, it >>> feels like things are somewhat arbitrarily enforced, which is probably very >>> frustrating for people trying to commit work/tickets. >>> >>> - Jeff >>> >>> >>> >>> On Mon, Feb 20, 2023 at 11:48 AM Dinesh Joshi <djo...@apache.org> wrote: >>> >>>> I’m a big fan of maintaining backward compatibility. Downgradability >>>> implies that we could potentially roll back an upgrade at any time. While I >>>> don’t think we need to retain the ability to downgrade in perpetuity it >>>> would be a good objective to maintain strict backward compatibility and >>>> therefore downgradability until a certain point. This would imply >>>> versioning metadata and extending it in such a way that prior version(s) >>>> could continue functioning. This can certainly be expensive to implement >>>> and might bloat on-disk storage. However, we could always offer an option >>>> for the operator to optimize the on-disk structures for the current version >>>> then we can rewrite them in the latest version. This optimizes the storage >>>> and opens up new functionality. This means new features that can work with >>>> old on-disk structures will be available while others that strictly require >>>> new versions of the data structures will be unavailable until the operator >>>> migrates to the new version. This migration IMO should be irreversible. >>>> Beyond this point the operator will lose the ability to downgrade which is >>>> ok. >>>> >>>> Dinesh >>>> >>>> On Feb 20, 2023, at 10:40 AM, Jake Luciani <jak...@gmail.com> wrote: >>>> >>>> >>>> There has been progress on >>>> >>>> https://issues.apache.org/jira/plugins/servlet/mobile#issue/CASSANDRA-8928 >>>> >>>> Which is similar to what datastax does for DSE. Would this be an >>>> acceptable solution? >>>> >>>> Jake >>>> >>>> On Mon, Feb 20, 2023 at 11:17 AM guo Maxwell <cclive1...@gmail.com> >>>> wrote: >>>> >>>>> It seems “An alternative solution is to implement/complete >>>>> CASSANDRA-8110 <https://issues.apache.org/jira/browse/CASSANDRA-8110>” >>>>> can give us more options if it is finished😉 >>>>> >>>>> Branimir Lambov <blam...@apache.org>于2023年2月20日 周一下午11:03写道： >>>>> >>>>>> Hi everyone, >>>>>> >>>>>> There has been a discussion lately about changes to the sstable >>>>>> format in the context of being able to abort a cluster upgrade, and the >>>>>> fact that changes to sstables can prevent downgraded nodes from reading >>>>>> any >>>>>> data written during their temporary operation with the new version. >>>>>> >>>>>> Most of the discussion is in CASSANDRA-18134 >>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-18134>, and is >>>>>> spreading into CASSANDRA-14277 >>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-14227> and >>>>>> CASSANDRA-17698 >>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-17698>, none of >>>>>> which is a good place to discuss the topic seriously. >>>>>> >>>>>> Downgradability is a worthy goal and is listed in the current >>>>>> roadmap. I would like to open a discussion here on how it would be >>>>>> achieved. >>>>>> >>>>>> My understanding of what has been suggested so far translates to: >>>>>> - avoid changes to sstable formats; >>>>>> - if there are changes, implement them in a way that is >>>>>> backwards-compatible, e.g. by duplicating data, so that a new version is >>>>>> presented in a component or portion of a component that legacy nodes will >>>>>> not try to read; >>>>>> - if the latter is not feasible, make sure the changes are only >>>>>> applied if a feature flag has been enabled. >>>>>> >>>>>> To me this approach introduces several risks: >>>>>> - it bloats file and parsing complexity; >>>>>> - it discourages improvement (e.g. CASSANDRA-17698 is no longer a LHF >>>>>> ticket once this requirement is in place); >>>>>> - it needs care to avoid risky solutions to address technical issues >>>>>> with the format versioning (e.g. staying on n-versions for 5.0 and >>>>>> needing >>>>>> a bump for a 4.1 bugfix might require porting over support for new >>>>>> features); >>>>>> - it requires separate and uncoordinated solutions to the problem and >>>>>> switching mechanisms for each individual change. >>>>>> >>>>>> An alternative solution is to implement/complete CASSANDRA-8110 >>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-8110>, which >>>>>> provides a method of writing sstables for a target version. During >>>>>> upgrades, a node could be set to produce sstables corresponding to the >>>>>> older version, and there is a very straightforward way to implement >>>>>> modifications to formats like the tickets above to conform to its >>>>>> requirements. >>>>>> >>>>>> What do people think should be the way forward? >>>>>> >>>>>> Regards, >>>>>> Branimir >>>>>> >>>>>> >>>>>> -- >>>>> you are the apple of my eye ! >>>>> >>>> -- >>>> http://twitter.com/tjake >>>> >>>>
