> > TL;DR - in progress migration off 2.2 to 5.0 is annoying as there were >> different bugs in the past we have to support again. Out of process >> migration to me feels far more plausible, but feels annoying without >> splitting off our reader/writer… doable… just more annoying… > > This is your main blocker for the 5.0/trunk code converting 2.2 sstables correctly. You would have to bring back a bunch of code paths special casing those versions and dealing with bugs in them. If you are actually interested in figuring out an offline 2.2 to 5.0 path I would recommend you do it in two steps. Offline 2.2 to 4.1? or what ever the latest thing is that can sstableupgrade a 2.2 sstable, and then offline the result of that with 5.0. But at that point maybe you just do it online in those two steps.
On Dec 12, 2024 at 11:34:31 AM, Štefan Miklošovič <smikloso...@apache.org> wrote: > I think it does not make a lot of sense to get away from Ant unless we > split it into more jars. Splitting it into more jars while moving away from > Ant at the same time is just too much work. So, what is the point of having > monolithic cassandra-all in Gradle / Maven? Smoother release? We mastered > that already. We are releasing, aren't we? Dependencies? That is working > too. Sure cache and all the "hacks" would go away overnight but otherwise > ... I think modularising it first so it is easier to reuse and so on is > more important. > > On Thu, Dec 12, 2024 at 6:21 PM David Capwell <dcapw...@apple.com> wrote: > >> but I still find myself very rarely interacting with ant >> >> >> I think that is where most people are as not many actually maintain or >> modify ant… there are so many things that bug me (lack of cache, making >> sure new people use the right version (was totally fun to learn 4.1 didn’t >> build with the default ant that got installed when I was helping out a new >> higher… we had to downgrade… woohoo….), hand rolled IDE integration, etc…), >> but I would disagree that ant is a blocker for different jars… we could >> switch off ant for Makefile, or bash, and we would still be able to produce >> new jars… Its more of… how many people actually feel comfortable enough to >> alter our build system to make such a change? I don’t… >> >> Our build system does not impact our ability to offer migration from 2.2 >> to 5.0… so don’t want to keep distracting this thread… >> >> TL;DR - in progress migration off 2.2 to 5.0 is annoying as there were >> different bugs in the past we have to support again. Out of process >> migration to me feels far more plausible, but feels annoying without >> splitting off our reader/writer… doable… just more annoying... >> >> On Dec 12, 2024, at 9:04 AM, Alex Petrov <al...@coffeenco.de> wrote: >> >> > I have for a while advocated for a shared lib to also share between >> Harry, accord, dtests etc >> >> Big +1 for a shared lib for our concurrency and test utils. Been >> intending to start working on this for a while now, but never got to do >> this so far. >> >> On Thu, Dec 12, 2024, at 5:58 PM, Benedict wrote: >> >> >> Why would ant get in the way? We already build multiple jars, and accord >> will be a submodule. We have far more organisational issues to overcome >> than ant. >> >> I have for a while advocated for a shared lib to also share between >> Harry, accord, dtests etc >> >> I am however not 100% sure about splitting read/write path, at least not >> as first posited. The idea of maintaining it as an API for dropping in >> different jars is a whole other world of potential pain I don’t want to >> countenance. Supporting eg bulk readers or writers or other integrations >> seems pretty feasible though. >> >> >> On 12 Dec 2024, at 16:53, Paulo Motta <pa...@apache.org> wrote: >> >> >> > I think that will not happen until we are out of Ant as doing this >> multi jar / subproject mumbo jumbo is not too much appealing to ... anybody? >> >> This is a contentious/controversial topic, but the more I work with >> gradle the more I lean towards ant's simplicity. That said, I'd support >> moving away if it becomes a technical blocker to break up cassandra-all - >> and if this happen I would vote for maven as replacement. :-D >> >> On Thu, Dec 12, 2024 at 11:42 AM Miklosovic, Stefan via dev < >> dev@cassandra.apache.org> wrote: >> >> These are all good ideas but in practical terms I think that will not >> happen until we are out of Ant as doing this multi jar / subproject mumbo >> jumbo is not too much appealing to ... anybody? >> >> ________________________________________ >> From: Paulo Motta <pa...@apache.org> >> Sent: Thursday, December 12, 2024 17:35 >> To: dev@cassandra.apache.org >> Subject: Re: Supporting 2.2 -> 5.0 upgrades >> >> EXTERNAL EMAIL - USE CAUTION when clicking links or attachments >> >> >> >> > +1 on moving the read/write logic into its own jar. >> >> +1, not only read-write logic but anything used by both the server and >> subprojects (ie. cassandra-sidecar), for example JMX Mbeans and other >> interfaces. >> >> I think one way to do that would be to split cassandra-all into >> cassandra-server and cassandra-common (anything used by both subprojects >> and server), but not sure if this would be feasible or what it would take. >> >> If there's loose agreement this would be a feasible path I'd be happy to >> create a JIRA to investigate what this would take. >> >> On Thu, Dec 12, 2024 at 11:26 AM Doug Rohrer <droh...@apple.com<mailto: >> droh...@apple.com>> wrote: >> +1 on moving the read/write logic into its own jar. >> >> Doug >> >> > On Dec 11, 2024, at 7:21 PM, David Capwell <dcapw...@apple.com<mailto: >> dcapw...@apple.com>> wrote: >> > >> > From a disk format point of view the only thing I remember was the disk >> type bug with UDTs. Bringing that logic back was hard as the type system >> (in 5.0) tries to avoid allowing construction of invalid states, and we >> would need to weaken that in order to enable the migration. Assuming the >> user migrated from 3.x to 4.x then the sstable metadata should have been >> rewritten to fix this bug. >> > >> > One thought (though know its a ton of effort).. we have talked about >> for a long time about moving the reading/writing logic into its jar (so >> tools don’t need cassandra-all and can limit the dependencies)… if we did >> that we could try to solve this as an out of process migration… have the >> 2.2 reader then write using 6.0 writer (ignoring compact storage… )… >> > >> >> On Dec 11, 2024, at 4:59 AM, Benedict <bened...@apache.org<mailto: >> bened...@apache.org>> wrote: >> >> >> >> I think 3.11 supported upgrade from 2.2, but I haven’t checked. I am >> fairly sure 4.x supported upgrade from 3.0.x also. >> >> >> >> >> >>> On 11 Dec 2024, at 12:53, Miklosovic, Stefan via dev < >> dev@cassandra.apache.org<mailto:dev@cassandra.apache.org>> wrote: >> >>> >> >>> I see. That makes sense. I think that by 3.x you meant basically the >> latest 3.11, right? I guess 2.2 -> 3.0 already works, we would just try to >> support 2.2 -> 3.11 straight away. I need to check where we are at in that >> area. >> >>> >> >>> ________________________________________ >> >>> From: Benedict <bened...@apache.org<mailto:bened...@apache.org>> >> >>> Sent: Wednesday, December 11, 2024 13:09 >> >>> To: dev@cassandra.apache.org<mailto:dev@cassandra.apache.org> >> >>> Cc: Miklosovic, Stefan; dev@cassandra.apache.org<mailto: >> dev@cassandra.apache.org>; Miklosovic, Stefan >> >>> Subject: Re: Supporting 2.2 -> 5.0 upgrades >> >>> >> >>> EXTERNAL EMAIL - USE CAUTION when clicking links or attachments >> >>> >> >>> >> >>> >> >>> >> >>> 2.2 is particularly hard because of the major storage format changes >> that took place. >> >>> >> >>> I think if we want to retain (restore) upgrade support from 3.x I >> would support that, but 2.x is probably too burdensome and likely to have >> too many hard edges. >> >>> >> >>> I think if users only had to upgrade 2.2->3.x then eg 3.x->6.0 that >> would be a pretty friendly upgrade path all things considered. >> >>> >> >>>> On 11 Dec 2024, at 12:03, Miklosovic, Stefan via dev < >> dev@cassandra.apache.org<mailto:dev@cassandra.apache.org>> wrote: >> >>>> >> >>>> Hey, >> >>>> >> >>>> I want to fork the thread where we are mentioning that 2.2 -> 5.0 >> would be cool to support. >> >>>> >> >>>> I was involved in checking that offline upgrades from 3.0 to 5.0 >> work and fixed few issues along the way (1), hence I can imagine that >> supporting 2.2 -> 5.0 would be basically the same thing just on steroids >> and more involved? Anyway, having a stab into this is not useless at all, I >> will at least go deep into the upgrade stuff I have never given a lot of >> thought to which is good learning experience. >> >>>> >> >>>> Any tips where to start? Was any progress done by anybody already in >> this matter to not start from zero? >> >>>> >> >>>> (1) >> https://urldefense.com/v3/__https://issues.apache.org/jira/browse/CASSANDRA-19002__;!!Nhn8V6BzJA!RFZoz6sQSrP_qLd0K_eNWO3UAc1s8mTT5SkFalUMwM7_l9gWfb4cnfTFvdY68zsh5-REW7T8ALTPQwqMM_gWWSyp$ >> >>>> >> >>>> Regards >> >>> >> >> >> > >> >> >>
