Re: Riak 1.3.1 high swap usage

2015-02-19 Thread Daniel Iwan
I absolutely agree. That is why we've change the setting vm.swappiness to 1 so it swaps only when absolutely necessary. I think we underestimated how much swap may be needed, but I also don't understand why so much hungry on memory. Is there a particular activity, like 2i queries, AAE or levelDB c

Re: Riak 1.3.1 high swap usage

2015-02-19 Thread Sean Cribbs
Daniel, Our tuning guides specifically recommend disabling swap. If your machine is so memory-starved that it needs to use swap during normal Riak operation, having it on is only going to make things worse. http://docs.basho.com/riak/latest/ops/tuning/linux/#Storage-and-File-System-Tuning On Thu

Re: Riak 1.3.1 high swap usage

2015-02-19 Thread Daniel Iwan
We are using levelDB as backend without any tuning. Also we are aware that performance may suffer due to potentially storing some of the copies (n=3) twice on the server. We are not so much concerned about latencies caused by that. What is worrying though is almost unbounded growth of swap used, wh

Re: Riak 1.3.1 high swap usage

2015-02-19 Thread John Daily
Daniel, you may be aware of this, but a 3-node Riak cluster is not recommended and may be playing a minor role in your resource problems. Every request will hit every server (except for some requests that are being made twice against a single server, making I/O that much worse), and depending on yo

Re: Riak 1.3.1 high swap usage

2015-02-19 Thread Christopher Meiklejohn
> On Feb 19, 2015, at 9:05 AM, Daniel Iwan wrote: > > Hi > On 3 node cluster Ubuntu 12.04, nodes 8GB RAM all nodes show 6GB taken > beam.smp, 2GB by our process. > beam started swapping and currently is using 23GB of swap space. > vm.swappiness is set to 1 > We are using ring 128. /var/lib/ria