Node leaving but waiting for handoff

2014-07-25 Thread marcel.koopman
Hi,

I've asked a node to leave the cluster, it did do a transfer to another
node, but now its not doing anything at all. 

What is riak doing now, why is the handoff waiting, can I influence this? 

== Unreachable Nodes
==
All nodes are up and reachable

root@77-72-144-214:~# riak-admin transfers
'riak@77.72.144.214' waiting to handoff 30 partitions

Active Transfers:
--




--
View this message in context: 
http://riak-users.197444.n3.nabble.com/Node-leaving-but-waiting-for-handoff-tp4031453.html
Sent from the Riak Users mailing list archive at Nabble.com.

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


Did a force-remove of two nodes, now system is unresponsive

2014-08-18 Thread marcel.koopman
We have a 5 node riak cluster.
Two nodes in this cluster, had to be removed because they are no longer
available (since a half year).
So a force remove was done.

After this, the 3 remaining nodes began to transfer all data. So we ended up
with a complete unresponsive system. The iowait is blocking us now.
So we are hoping that this will settle today, the next transaction was
actually adding two new nodes.

And yes this is production, Is there any chance we lost data?



--
View this message in context: 
http://riak-users.197444.n3.nabble.com/Did-a-force-remove-of-two-nodes-now-system-is-unresponsive-tp4031603.html
Sent from the Riak Users mailing list archive at Nabble.com.

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


Re: Did a force-remove of two nodes, now system is unresponsive

2014-08-19 Thread marcel.koopman
Hi,

The configured ring size is 64 (default i believe).
Is this something we can change now?
Also still riak is doing transfers, making some of the nodes unresponsive.
We've limited the transfers to 1.
But how do we handle this?

Kind regards,
Marcel


2014-08-19 8:58 GMT+02:00 Ciprian Manea [via Riak Users] <
ml-node+s197444n4031634...@n3.nabble.com>:

> Hi Marcel,
>
> What is the configured ring size for this cluster?
>
> You can slow down the transfers by running $ riak-admin transfer-limit 1
> in one of your riak nodes. iowait should decrease as well once
> transfer-limit is lowered, unless one of your disks is failing or is about
> to fail.
>
>
> Regards,
> Ciprian
>
>
> On Mon, Aug 18, 2014 at 9:18 PM, marcel.koopman <[hidden email]
> <http://user/SendEmail.jtp?type=node&node=4031634&i=0>> wrote:
>
>> We have a 5 node riak cluster.
>> Two nodes in this cluster, had to be removed because they are no longer
>> available (since a half year).
>> So a force remove was done.
>>
>> After this, the 3 remaining nodes began to transfer all data. So we ended
>> up
>> with a complete unresponsive system. The iowait is blocking us now.
>> So we are hoping that this will settle today, the next transaction was
>> actually adding two new nodes.
>>
>> And yes this is production, Is there any chance we lost data?
>>
>>
>>
>> --
>> View this message in context:
>> http://riak-users.197444.n3.nabble.com/Did-a-force-remove-of-two-nodes-now-system-is-unresponsive-tp4031603.html
>> Sent from the Riak Users mailing list archive at Nabble.com.
>>
>> ___
>> riak-users mailing list
>> [hidden email] <http://user/SendEmail.jtp?type=node&node=4031634&i=1>
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>
>
> ___
> riak-users mailing list
> [hidden email] <http://user/SendEmail.jtp?type=node&node=4031634&i=2>
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
> --
>  If you reply to this email, your message will be added to the discussion
> below:
>
> http://riak-users.197444.n3.nabble.com/Did-a-force-remove-of-two-nodes-now-system-is-unresponsive-tp4031603p4031634.html
>  To unsubscribe from Did a force-remove of two nodes, now system is
> unresponsive, click here
> <http://riak-users.197444.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4031603&code=bWFyY2VsLmtvb3BtYW5AZ21haWwuY29tfDQwMzE2MDN8LTE3MDk3MTY2NDg=>
> .
> NAML
> <http://riak-users.197444.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>



-- 
Met vriendelijke groet,

Marcel Koopman




--
View this message in context: 
http://riak-users.197444.n3.nabble.com/Did-a-force-remove-of-two-nodes-now-system-is-unresponsive-tp4031603p4031635.html
Sent from the Riak Users mailing list archive at Nabble.com.___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com