Hi Germain, It's difficult to say if 3 GB of RAM is not enough. Bitcask maintains an in memory structure called a keydir. The bitcask intro describes the keydir as follows:
"A keydir is simply a hash table that maps every key in a Bitcask to a fixed-size structure giving the file, offset, and size of the most recently written entry for that key" This means that the amount of memory used by Riak depends on how many keys you have and the size of each key. So if you have lots of keys you may run out memory. How much is the riak process consuming before you request the key list for the bucket? Thanks, Dan Daniel Reverri Developer Advocate Basho Technologies, Inc. d...@basho.com On Thu, Jun 24, 2010 at 8:31 AM, Germain Maurice < germain.maur...@linkfluence.net> wrote: > Dan, > > I switched back to Bitcask, i have a n_value = 2 bucket over two nodes > (each node has 3GB RAM) > > i requested : > curl > "http://10.0.0.40:8098/riak/test?keys=stream";<http://10.0.0.40:8098/riak/test?keys=stream> > and i got this error > [...]"81902","50339","592","56815","55290","8857","251curl: (18) transfer > closed with outstanding read data remaining > > 54","12321","17879","39060","48516","79056","72858","17509","15484","14623","61998","96868","55237","8196", > > "68830","39850","44175","68814","18970","12112","15228","67859","24275","61334","67503","86341","75438","68686", > > "45040","10442","20863","22342","68666","86289","12816","22693","4919","90026","69932","24564","19181","50704", > > "59385","95929","34755","97493","7987","82073","70540","89454","44260","53609"]} > > The riak process "riak console" died. > > Have a look to the dmesg.txt attached with the message. > Can you certify me there is not enough RAM for riak ? > > > > Le 24/06/10 10:28, Germain Maurice a écrit : > > Dan, > > I don't know exactly when the errors occured because there are no > timestamps :( > I'm sure that's during a basho_bench test (no map-reduce was launched). I > see these errors on the all nodes of my cluster. > However, the riak nodes kept running, now i'm using riak 0.11.0 with > innostore, previously i led some tests with bitcask. I believe that's during > i was using Bitcask the errors occured. > Each node has 3GB memory installed, i assume this could be a bit small, but > why a crash ?! > > Thank you for the links, i just got a glance on them. > > > Le 23/06/10 19:37, Dan Reverri a écrit : > > Hi Germain, > > I'm sorry if I've missed any background information that may have been > provided earlier but can you tell me the scenario that led up to these > errors? Are you running a load test against a cluster? Are you seeing these > out of memory errors in one of your nodes? How much memory does the machine > running the node have? What type of load is being placed on the node (get, > put, map reduce, etc.)? > > If you are using the Bitcask backend, which is default in the latest > release, you should be aware that all keys held on the node must fit in > memory (this does NOT include the values, just the keys). Here is a related > discussion from IRC: > http://gist.github.com/438065 > > Also worth reviewing is this discussion from the mailing list: > > http://riak.markmail.org/search/?q=bitcask%20key%20memory#query:bitcask%20key%20memory+page:1+mid:2rreshpri7laq57h+state:results > > Thank you, > Dan > > Daniel Reverri > Developer Advocate > Basho Technologies, Inc. > d...@basho.com > > > On Wed, Jun 23, 2010 at 12:24 PM, Germain Maurice < > germain.maur...@linkfluence.net> wrote: > >> I'm still disturbing you with errors... >> >> I just discovered some errors and i would like to know if it's normal... >> You can see the errors in the attached file. >> >> >> -- >> Germain Maurice >> Administrateur Système/Réseau >> Tel : +33.(0)1.42.43.54.33 >> >> http://www.linkfluence.net >> >> >> _______________________________________________ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> > > > -- > Germain Maurice > Administrateur Système/Réseau > Tel : +33.(0)1.42.43.54.33 > http://www.linkfluence.net > > > _______________________________________________ > riak-users mailing > listriak-us...@lists.basho.comhttp://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > > -- > Germain Maurice > Administrateur Système/Réseau > Tel : +33.(0)1.42.43.54.33 > http://www.linkfluence.net > > > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > >
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
