That's what I'd be doing, but I wouldn't expect it to run for 3 days this time. My guess is that whatever was going wrong with the bootstrap when you had 3 nodes starting at once was interfering with the completion of the 1 remaining node of those 3. A clean bootstrap of a single node should complete eventually, and I would think it'll be a lot less than 3 days. Our database is much smaller than yours at the moment, so I can't really guide you on how long it should take, but I'd think that others on the list with similar database sizes might be able to give you a better idea.
Steve On Fri, Apr 18, 2014 at 1:43 PM, Phil Burress <philburress...@gmail.com>wrote: > First, I just stopped 2 of the nodes and left one running. But this > morning, I stopped that third node, cleared out the data, restarted and let > it rejoin again. It appears streaming is done (according to netstats), > right now it appears to be running compaction and building secondary index > (according to compactionstats). Just sit and wait I guess? > > > On Fri, Apr 18, 2014 at 2:23 PM, Steven A Robenalt > <srobe...@stanford.edu>wrote: > >> Looking back through this email chain, it looks like Phil said he wasn't >> using vnodes. >> >> For the record, we are using vnodes since we brought up our first >> cluster, and have not seen any issues with bootstrapping new nodes either >> to replace existing nodes, or to grow/shrink the cluster. We did adhere to >> the caveats that new nodes should not be seed nodes, and that we should >> allow each node to join the cluster completely before making any other >> changes. >> >> Phil, when you dropped to adding just the single node to your cluster, >> did you start over with the newly added node (blowing away the database >> created on the previous startup), or did you shut down the other 2 added >> nodes and leave the remaining one in progress to continue? >> >> Steve >> >> >> On Fri, Apr 18, 2014 at 10:40 AM, Robert Coli <rc...@eventbrite.com>wrote: >> >>> On Fri, Apr 18, 2014 at 5:05 AM, Phil Burress >>> <philburress...@gmail.com>wrote: >>> >>>> nodetool netstats shows 84 files. They are all at 100%. Nothing showing >>>> in Pending or Active for Read Repair Stats. >>>> >>>> I'm assuming this means it's done. But it still shows "JOINING". Is >>>> there an undocumented step I'm missing here? This whole process seems >>>> broken to me. >>>> >>> >>> Lately it seems like a lot more people than usual are : >>> >>> 1) using vnodes >>> 2) unable to bootstrap new nodes >>> >>> If I were you, I would likely file a JIRA detailing your negative >>> experience with this core functionality. >>> >>> =Rob >>> >>> >>> >> >> >> >> -- >> Steve Robenalt >> Software Architect >> HighWire | Stanford University >> 425 Broadway St, Redwood City, CA 94063 >> >> srobe...@stanford.edu >> http://highwire.stanford.edu >> >> >> >> >> >> > -- Steve Robenalt Software Architect HighWire | Stanford University 425 Broadway St, Redwood City, CA 94063 srobe...@stanford.edu http://highwire.stanford.edu
