But I didn't restart the red one. On Thu, Apr 1, 2010 at 12:18 PM, Jonathan Ellis <jbel...@gmail.com> wrote:
> There shouldn't be anything to clean up. (The temporary streaming > files it anticompacted are automatically removed on restart) > > On Thu, Apr 1, 2010 at 2:17 PM, Dan Di Spaltro <dan.dispal...@gmail.com> > wrote: > > Okay, so should I run any more commands like cleanup before? > > > > On Thu, Apr 1, 2010 at 12:09 PM, Jonathan Ellis <jbel...@gmail.com> > wrote: > >> > >> Bootstrap source restarting will always fail bootstrap. You'll need > >> to restart the blue one too now, I'm afraid. > >> > >> On Thu, Apr 1, 2010 at 2:01 PM, Dan Di Spaltro <dan.dispal...@gmail.com > > > >> wrote: > >> > Before the Red one rebooted it had 1 active STREAM-STAGE. Now it has > 0 > >> > in > >> > STREAM-STAGE. > >> > > >> > On Thu, Apr 1, 2010 at 11:57 AM, Dan Di Spaltro > >> > <dan.dispal...@gmail.com> > >> > wrote: > >> >> > >> >> Red one. > >> >> Gary - both say nothing is happening with no destinations or sources. > >> >> > >> >> On Thu, Apr 1, 2010 at 11:55 AM, Jonathan Ellis <jbel...@gmail.com> > >> >> wrote: > >> >>> > >> >>> which node rebooted, the red one, or the blue one? > >> >>> > >> >>> On Thu, Apr 1, 2010 at 11:26 AM, Dan Di Spaltro > >> >>> <dan.dispal...@gmail.com> > >> >>> wrote: > >> >>> > So we are adding another node to the cluster with the latest 0.6 > >> >>> > branch > >> >>> > (RC1). It seems to be hung in some limbo state. > >> >>> > Before bootstrapping our cluster had 50-60GB spread fairly evenly > >> >>> > across 4 > >> >>> > machines, with RF=3. One machine had more load than the others, > >> >>> > and > >> >>> > sure > >> >>> > enough bootstrapping selected that node. That is the red > machine. > >> >>> > The > >> >>> > light blue machine is the new machine. > >> >>> > I have attached a graph to illustrate when the bootstrap process > >> >>> > started. > >> >>> > In jconsole the streamingservice status was "performing > >> >>> > anticompaction..." > >> >>> > for over 18-24 hrs. It is currently in "nothing is happening". > It > >> >>> > did > >> >>> > have 1 active STREAM-STAGE task, but the machine had to be > rebooted > >> >>> > for > >> >>> > something unrelated to cassandra. Now the light blue machine > appears > >> >>> > to > >> >>> > be > >> >>> > getting data, but its growing at virtually the same rate as the > >> >>> > other > >> >>> > machines which makes me think it is part of the cluster and not > >> >>> > actually > >> >>> > streaming data from the machine its supposed to. > >> >>> > Any other ideas on how to debug? > >> >>> > > >> >>> > -- > >> >>> > Dan Di Spaltro > >> >>> > > >> >> > >> >> > >> >> > >> >> -- > >> >> Dan Di Spaltro > >> > > >> > > >> > > >> > -- > >> > Dan Di Spaltro > >> > > > > > > > > > -- > > Dan Di Spaltro > > > -- Dan Di Spaltro
