Re: Riak Memory Bloat issues with RiakCS/BitCask

2013-08-20 Thread Evan Vigil-McClanahan
IIRC cs stores ~3x1mb entries for each 1mb of object that you store at default n_val. so, 300k * 3 * 9 = 810k, * 150b (to be conservative) = 1.1 GB. There will be some other static and per-entry overheads, but you should be able to do better than that before you OOM, even on a 1 node system. Mi

Re: Riak Memory Bloat issues with RiakCS/BitCask

2013-08-20 Thread Idan Shinberg
Thank you all for you kind and quick answers However , even on a 3 node or 5 node cluster We're still seeing memory bloat ( only much notably slower , as load is distributed between more machines ) it's important to stress , this is an "read-append" only cluster - This means the data never expire

RE: Riak Memory Bloat issues with RiakCS/BitCask

2013-08-20 Thread Erik Søe Sørensen
Your max file size is (far!) less than your small file size threshold - which means that at each merge, *all* of the files will participate in the merge. No wonder you need a lot of simultaneously open files... and long merge times too, of course. Try changing these parameters. Opri

Re: Riak Memory Bloat issues with RiakCS/BitCask

2013-08-19 Thread Alexander Sicular
Just stop there. You got all your replicas (and fd's) on one machine. Get at least 3 nodes in your cluster. -Alexander @siculars http://siculars.posthaven.com Sent from my iRotaryPhone On Aug 18, 2013, at 19:32, Idan Shinberg wrote: > We have a ~300GB Riak Single Node Cluster _