Hello, I just started the rolling upgrade procedure from 1.1.10 to 2.1.10. Our strategy is to simultaneously upgrade one server from each replication group. So, if we have a 6 nodes with RF=2, we will upgrade 3 nodes at a time (from distinct replication groups).
My question is: do the newly upgraded nodes show as "Down" in the "nodetool ring" of the old cluster (1.1.10)? Because I thought that network compatibility meant nodes from a newer version would receive traffic (write + reads) from the previous version without problems. Cheers, Paulo 2013/9/26 Paulo Motta <pauloricard...@gmail.com> > Hello Charles, > > Thank you very much for your detailed upgrade report. It'll be very > helpful during our upgrade operation (even though we'll do a rolling > production upgrade). > > I'll also share our findings during the upgrade here. > > Cheers, > > Paulo > > > 2013/9/24 Charles Brophy <cbro...@zulily.com> > >> Hi Paulo, >> >> I just completed a migration from 1.1.10 to 1.2.10 and it was >> surprisingly painless. >> >> The course of action that I took: >> 1) describe cluster - make sure all nodes are on the same schema >> 2) shutoff all maintenance tasks; i.e. make sure no scheduled repair is >> going to kick off in the middle of what you're doing >> 3) snapshot - maybe not necessary but it's so quick it makes no sense to >> skip this step >> 4) drain the nodes - I shut down the entire cluster rather than chance >> any incompatible gossip concerns that might come from a rolling upgrade. I >> have the luxury of controlling both the providers and consumers of our >> data, so this wasn't so disruptive for us. >> 5) Upgrade the nodes, turn them on one-by-one, monitor the logs for funny >> business. >> 6) nodetool upgradesstables >> 7) Turn various maintenance tasks back on, etc. >> >> The worst part was managing the yaml/config changes between the versions. >> It wasn't horrible, but the diff was "noisier" than a more incremental >> upgrade typically is. A few things I recall that were special: >> 1) Since you have an existing cluster, you'll probably need to set the >> default partitioner back to RandomPartitioner in cassandra.yaml. I believe >> that is outlined in NEWS. >> 2) I set the initial tokens to be the same as what the nodes held >> previously. >> 3) The timeout is now divided into more atomic settings and you get to >> decided how (or if) to configure it from the default appropriately. >> >> tldr; I did a standard upgrade and payed careful attention to the >> NEWS.txt upgrade notices. I did a full cluster restart and NOT a rolling >> upgrade. It went without a hitch. >> >> Charles >> >> >> >> >> >> >> On Tue, Sep 24, 2013 at 2:33 PM, Paulo Motta <pauloricard...@gmail.com>wrote: >> >>> Cool, sounds fair enough. Thanks for the help, Rob! >>> >>> If anyone has upgraded from 1.1.X to 1.2.X, please feel invited to share >>> any tips on issues you're encountered that are not yet documented. >>> >>> Cheers, >>> >>> Paulo >>> >>> >>> 2013/9/24 Robert Coli <rc...@eventbrite.com> >>> >>>> On Tue, Sep 24, 2013 at 1:41 PM, Paulo Motta >>>> <pauloricard...@gmail.com>wrote: >>>> >>>>> Doesn't the probability of something going wrong increases as the gap >>>>> between the versions increase? So, using this reasoning, upgrading from >>>>> 1.1.10 to 1.2.6 would have less chance of something going wrong then from >>>>> 1.1.10 to 1.2.9 or 1.2.10. >>>>> >>>> >>>> Sorta, but sorta not. >>>> >>>> https://github.com/apache/cassandra/blob/trunk/NEWS.txt >>>> >>>> Is the canonical source of concerns on upgrade. There are a few cases >>>> where upgrading to the "root" of X.Y.Z creates issues that do not exist if >>>> you upgrade to the "head" of that line. AFAIK there have been no cases >>>> where upgrading to the "head" of a line (where that line is mature, like >>>> 1.2.10) has created problems which would have been avoided by upgrading to >>>> the "root" first. >>>> >>>> >>>>> I'm hoping this reasoning is wrong and I can update directly from >>>>> 1.1.10 to 1.2.10. :-) >>>>> >>>> >>>> That's what I plan to do when we move to 1.2.X, FWIW. >>>> >>>> =Rob >>>> >>> >>> >>> >>> -- >>> Paulo Ricardo >>> >>> -- >>> European Master in Distributed Computing*** >>> Royal Institute of Technology - KTH >>> * >>> *Instituto Superior Técnico - IST* >>> *http://paulormg.com* >>> >> >> > > > -- > Paulo Ricardo > > -- > European Master in Distributed Computing*** > Royal Institute of Technology - KTH > * > *Instituto Superior Técnico - IST* > *http://paulormg.com* > -- Paulo Ricardo -- European Master in Distributed Computing*** Royal Institute of Technology - KTH * *Instituto Superior Técnico - IST* *http://paulormg.com*
