Hi Anthony, 1) Which version of riak are you running? I think there are more messages in the mailing list about people who have had nodes get stuck leaving (it's worth also searching for "handoff" as that's the part of the vnode lifecycle which is the important one in a node leaving manoeuvre).
2) I think to play it safe, it's probably best to have the nodes leave one by one. I don't know exactly how riak keeps track of network partitions, but I could imagine it seeing 3 nodes dropping off as some kind of network partition rather than those 3 nodes wanting to leave. Hopefully this will get the ball rolling with getting your issue fixed. Sam On Wed, Apr 4, 2012 at 8:02 AM, Anthony Molinaro <antho...@alumni.caltech.edu> wrote: > Ping > > On Mon, Apr 02, 2012 at 06:28:00PM -0700, Anthony Molinaro wrote: >> Hi, >> >> So I found one of my riak clusters misbehaving and found out it was >> running on some inferior hardware. After getting some new machines from >> my ops group. I went ahead and added several nodes. Everything went >> relatively smoothly until I tried to have the old nodes leave the ring. >> >> I need to remove 4 nodes, and I assumed since I had been successful >> adding a few nodes at once, I would also be able to have multiple nodes >> leave. >> >> Unfortunately, that does not appear to be the case. One node (the first >> removed) completely left, the others did not. Then no matter how many >> times I did a 'riak-admin leave' they would not leave. I re-added the >> one that had left successfully to see if a ring change would unstick >> something. It does appear to, as even with all the original nodes back >> the 3 I attempted to have leave are still not leaving. >> >> I'm going through now and just doing a rolling restart hoping that >> might fix things, but I'm thinking I might be stuck. >> >> Is there anyway to remove these nodes? >> >> Thanks, >> >> -Anthony >> >> -- >> ------------------------------------------------------------------------ >> Anthony Molinaro <antho...@alumni.caltech.edu> >> >> _______________________________________________ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > -- > ------------------------------------------------------------------------ > Anthony Molinaro <antho...@alumni.caltech.edu> > > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com -- Samuel Elliott s...@lenary.co.uk http://lenary.co.uk/ +44 (0)7891 993 664 _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
