RE: High disk usage on node

2018-11-01 Thread Travis Kirstine
Fred / Nicholas, Thanks for your input! We are running riak (leveldb with AAE) inside of VMs, as a result we may be able to simply copy the VM to a new server and add more storage to the logical volume. So our plan would be to: - Shutdown the failed node - Copy the VM and

Re: High disk usage on node

2018-11-01 Thread Fred Dushin
I think your best bet is to do a force-replace (and then a manual repair, if you are not using AAE) with a node that has higher capacity than your current standby. You are correct that replacing with your standby will fail when you run repairs and end up running out of space. I think you do NO

RE: High disk usage on node

2018-11-01 Thread Nicholas Adams
Hi Travis, I see you have encountered the disk space pickle. Just for the record, the safest way to run Riak in production is to keep all resources (CPU, RAM, Network, Disk Space, I/O etc.) below 70% utilization on all nodes at all times. The reason behind this is to compensate for when one or m

High disk usage on node

2018-11-01 Thread Travis Kirstine
I'm running riak (v2.14) in a 5 node cluster and for some reason one of the nodes has higher disk usage than the other nodes. The problem seems to be related to how riak distributes the partitions, in my case I'm using the default 64, riak has given each node 12 partition except one node that g