Re: LevelDB memory problem crashes Erlang

2012-08-10 Thread Jared Morrow
The information Joseph requested is important though. The number of improvements to LevelDB in Riak 1.2 was huge, so knowing if this is Riak 1.2 or Riak 1.1.4 would be great. -Jared On Aug 10, 2012, at 7:45 PM, Philipp Borgers wrote: > Seems like you don't have enough RAM. If you do a lot of

Re: LevelDB memory problem crashes Erlang

2012-08-10 Thread Philipp Borgers
Seems like you don't have enough RAM. If you do a lot of inserts this seems to be normal. I had this issue while benchmarking riak with basho_bench. You should add limit for the requests per second in your script. Take a look at the memory profile of the erlang process (htop, dstat) while importing

inaccurate documentation?

2012-08-10 Thread Jonathan Tsai
Hello, New to the community and mailing list here. I was stuck for a while on Part 2 of the Riak Fast Track (http://wiki.basho.com/Building-a-Development-Environment.html). It seems like the `make devrel` step can only be completed if installing from the GitHub source. I also tried Debian/Ubun

Re: LevelDB memory problem crashes Erlang

2012-08-10 Thread Joseph Blomstedt
Leo, This sounds like a tuning issue, the defaults for LevelDB (esp. on Riak 1.2) use up a decent amount of RAM. There should be upcoming documentation on all of this, but things are still in the works. In any case, we should be able to help you out here on the mailing list. A few questions: 1. W

LevelDB memory problem crashes Erlang

2012-08-10 Thread leonickel
Hi all, I'm facing a weird problem using eleveldb store option data. The amount of memory starts to increase while i'm running a script to put data inside riak, nearly 10 million of keys in a single bucket and suddenly riak process crashes. My object is in json format and is about 1.5 kb each. At

Reasons for using gen_server to gather statictics with folsom

2012-08-10 Thread Zhemzhitsky Sergey
Hi riak gurus, Recently riak 1.2 has been released that uses folsom library to gather statistics. I’d like to use the same library (folsom) in my application so could you answer the following questions: 1. Why separate gen_servers (riak_api_stat, riak_core_stat, riak_kv_stat) were used

Re: multi-get (yet again)

2012-08-10 Thread Parnell Springmeyer
I'm interested in this, I'll fork the repo and see what I can get added in there. On Aug 10, 2012, at 7:52 AM, Bryan Fink wrote: > On Thu, Aug 9, 2012 at 5:11 AM, Kresten Krab Thorup wrote: >> The only issue with this approach is AFAIK that M/R effectively runs with >> R=1, i.e. it doesn't ens

Re: multi-get (yet again)

2012-08-10 Thread Bryan Fink
On Thu, Aug 9, 2012 at 5:11 AM, Kresten Krab Thorup wrote: > The only issue with this approach is AFAIK that M/R effectively runs with > R=1, i.e. it doesn't ensure that a value is consistent across replicas. > > IMHO riak_kv_mapreduce should have a map_get_object_value, which does a > proper Ri

multi_backend, multiple storage devices, LevelDB and its cache_size

2012-08-10 Thread David Sky
Hello, My setup is a 128 partitions running on 4 identical, commodity servers, each with 30GB of RAM and 4 HDD allocated for Riak. According to this page http://wiki.basho.com/LevelDB.html to calculate "cache_size" for my setup, I should do: ((26GB / 2) * (1024 ^ 3)) / (128 / 4) = 416MB I guess