I mean add an additional two nodes to my cluster and pointing them at the other nodes in the cluster, to handle data migration.
On Wed, Jul 1, 2015 at 11:40 AM, Jonathan Haddad <[email protected]> wrote: > When you say add 2 nodes, do you mean bootstrap, or upgrade in place? > > On Wed, Jul 1, 2015 at 11:37 AM David Aronchick <[email protected]> > wrote: > >> This helps - so let me understand: >> >> Starting point: >> - 4 nodes running 2.1.4 >> - System is healthy >> >> Decide to upgrade: >> - Add 2 nodes running 2.1.5 >> - Run nodetool upgradestable >> - Wait until system is healthy >> - Stop 2 nodes running 2.1.4 >> - Run nodetool upgradestable >> - Add 2 nodes running 2.1.5 >> - Run nodetool upgradestable >> - Stop 2 nodes running 2.1.4 >> - Run nodetool upgradestable >> >> Finished? >> >> On Wed, Jul 1, 2015 at 1:49 AM, Alain RODRIGUEZ <[email protected]> >> wrote: >> >>> Basically, when you add nodes, add them on the correct version to avoid >>> schema / network issues in your streams. Also, try to update all the node >>> using rolling restarts in a reduced time frame after stopping repairs, with >>> all the nodes up, etc. >>> >>> You must have a healthy cluster before performing an upgrade. >>> >>> Remember running a nodetool upgradesstable after any upgrade (if not >>> needed it will end fast, so I would run it always as a best practice, just >>> in case). >>> >>> C*heers, >>> >>> Alain >>> >>> 2015-07-01 2:16 GMT+02:00 David Aronchick <[email protected]>: >>> >>>> That is a GREAT lead! So it looks like I can't add a few nodes to the >>>> cluster of the new version, have it settle down, and then upgrade the rest? >>>> >>>> On Tue, Jun 30, 2015 at 11:58 AM, Alain RODRIGUEZ <[email protected]> >>>> wrote: >>>> >>>>> "Would it matter that I'm mixing cassandra versions? " >>>>> >>>>> From: >>>>> http://docs.datastax.com/en/upgrade/doc/upgrade/datastax_enterprise/upgrdLim.html >>>>> >>>>> "General upgrade limitations¶ >>>>> >>>>> Do not run nodetool repair. >>>>> Do not enable new features. >>>>> Do not issue these types of queries during a rolling restart: DDL, >>>>> TRUNCATE >>>>> *During upgrades, the nodes on different versions show a schema >>>>> disagreement*." >>>>> >>>>> I think this is a good lead. >>>>> >>>>> C*heers, >>>>> >>>>> Alain >>>>> >>>>> 2015-06-30 20:22 GMT+02:00 David Aronchick <[email protected]>: >>>>> >>>>>> I appreciate the thoughts! My issue is that it seems to work >>>>>> perfectly, until the node goes away. Would it matter that I'm mixing >>>>>> cassandra versions? (2.1.4 and 2.1.5)? >>>>>> >>>>>> On Tue, Jun 30, 2015 at 5:23 AM, Alain RODRIGUEZ <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi David ? >>>>>>> >>>>>>> What does a "nodetool describecluster" output look like ? >>>>>>> >>>>>>> My guess is you might be having a schema version desynchronisation. >>>>>>> If you see a node with different schema version you might want to try a >>>>>>> "nodetool resetlocal*schema* - Reset node's local *schema* >>>>>>> and resync" >>>>>>> >>>>>>> You asked for any thoughts, this is a thought, not sure if it will >>>>>>> help, I hope so. >>>>>>> >>>>>>> C*heers, >>>>>>> >>>>>>> Alain >>>>>>> >>>>>>> 2015-06-30 1:44 GMT+02:00 Robert Coli <[email protected]>: >>>>>>> >>>>>>>> On Mon, Jun 29, 2015 at 2:43 PM, David Aronchick < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Ping--- any thoughts here? >>>>>>>>> >>>>>>>> >>>>>>>> I don't have any thoughts on your specific issue at this time, but >>>>>>>> FWIW #cassandra on freenode is sometimes a better forum for interactive >>>>>>>> debugging of operational edge cases. >>>>>>>> >>>>>>>> =Rob >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >>
