The are several things to consider here: - You can't have DC of two nodes with RF=3... - Are you sure that new DC will handle all production traffic? - if new nodes much more powerful than other (memory/CPU/disk type) that could also cause unpredictable spikes when request will hit the "smaller" node.
I personally maybe would go following way (need to calculate how many joins/decommissions will be at the end): - Decommission one node from prod DC - Form new DC from two new machines and decommissioned one. - Rebuild DC from existing one, make sure that repair finished, etc. - Switch traffic - Remove old DC - Add nodes from old DC one by one into new DC Upgrade to Cassandra 4.0 should be done either prior to that, or after - you shouldn't do it when doing bootstrapping/decomissioning... On Sat, Mar 20, 2021 at 4:09 PM Lapo Luchini <l...@lapo.it> wrote: > I have a 6 nodes production cluster running 3.11.9 with the default > num_tokens=256… which is fine but I later discovered is a bit of a > hassle to do repairs and is probably better to lower that to 16. > > I'm adding two new nodes with much higher space storage and I was > wondering which migration strategy is better. > > If I got it correct I was thinking about this: > 1. add the 2 new nodes as a new "temporary DC", with num_token=16 RF=3 > 2. repair it all, then test it a bit > 3. switch production applications to "DC-temp" > 4. drop the old 6-node DC > 5. re-create it from scratch with num_token=16 RF=3 > 6. switch production applications to "main DC" again > 7. drop "DC-temp", eventually integrate nodes into "main DC" > > I'd also like to migrate from 3.11.9 to 4.0-beta2 (I'm running on > FreeBSD so those are the options), does it make sense to do it during > the mentioned "num_tokens migration" (at step 1, or 5) or does it make > more sense to do it at step 8, as a in-place rolling upgrade of each of > the 6 (or 8) nodes? > > Did I get it correctly? > Can it be done "better"? > > Thanks in advance for any suggestion or correction! > > -- > Lapo Luchini > l...@lapo.it > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org > For additional commands, e-mail: user-h...@cassandra.apache.org > > -- With best wishes, Alex Ott http://alexott.net/ Twitter: alexott_en (English), alexott (Russian)
