have a new node take over the role of a downed, unrecoverable node?

2010-10-16 Thread Jesse Newland
I'm running through some disaster scenarios before bringing a riak cluster into production, and have run into a scenario that I can't work through the proper resolution for just yet: Say an ec2 instance that was a part of a ring went away quickly, and data from it was unrecoverable. How might

Re: have a new node take over the role of a downed, unrecoverable node?

2010-10-16 Thread Sean Cribbs
`leave` is exactly what you want to do then. Once the old node has left (use `ringready` to track its exit), add the new neode. If the EBS volume containing the node's data was not lost, you could mount it onto the new node to save some recovery time, and then reip. However, you'll need to r

Re: have a new node take over the role of a downed, unrecoverable node?

2010-10-16 Thread Jesse Newland
The description of leave on the wiki mentions that it "causes the node to leave the cluster it participates in" - I assume "the node" refers to the node this command is run on? How would I "leave" a node that I can't run this command on anymore? Regards - Jesse Newland --- je...@railsmachine.c

Re: have a new node take over the role of a downed, unrecoverable node?

2010-10-16 Thread Alexander Sicular
This has come up before. "Leave" is what is currently available and needs to be run on the node that wants to leave. This, of course, means the node needs to be available. What you really want is a kick like "remove" or something that doesn't exist yet, afaik. I think there is a ticket open. -alex

Re: have a new node take over the role of a downed, unrecoverable node?

2010-10-16 Thread Sean Cribbs
Sorry, I wasn't completely clear. You can make any node "leave" from the console. e.g. riak_core_gossip:remove_from_cluster('r...@some-host.com'). Sean Cribbs Developer Advocate Basho Technologies, Inc. http://basho.com/ On Oct 16, 2010, at 5:05 PM, Alexander Sicular wrote: > This has come up

Re: have a new node take over the role of a downed, unrecoverable node?

2010-10-16 Thread Jesse Newland
Thanks Sean! Regards - Jesse Newland --- je...@railsmachine.com 404.216.1093 On Oct 16, 2010, at 7:01 PM, Sean Cribbs wrote: > Sorry, I wasn't completely clear. You can make any node "leave" from the > console. e.g. > > riak_core_gossip:remove_from_cluster('r...@some-host.com'). > > Sean Cri

Re: have a new node take over the role of a downed, unrecoverable node?

2010-10-16 Thread Leonid Riaboshtan
>> However, you'll need to reip on all machines. Hmm, isn't stuff like that should be treated automaticly by Riak? I mean I have a cluster where nodes leave, nodes come. And after each come/leave I need to do something to nodes in entire cluster to entroduce/remove new/old node and repartion the d

Re: have a new node take over the role of a downed, unrecoverable node?

2010-10-16 Thread Sean Cribbs
On Oct 16, 2010, at 9:52 PM, Leonid Riaboshtan wrote: > >> However, you'll need to reip on all machines. > > Hmm, isn't stuff like that should be treated automaticly by Riak? I mean I > have a cluster where nodes leave, nodes come. And after each come/leave I > need to do something to nodes in