Re: Store whole database in memory

2011-05-29 Thread Nico Meyer
Hi Michael, Greg's advice is probably the best, if you really always want to read back or update predefined groups of 1000 keys at once. It will increase the rate at which you can write and read by a factor of 1000 ;-). But if that's not what you want to do, and we really don't know what you

Re: Store whole database in memory

2011-05-28 Thread Greg Nelson
Depending on the n_val you have set for that bucket, Riak will store the objects n times on n different nodes. There are two other parameters you should know about, r and w. When writing, Riak will wait for w of the n nodes to finish the write before returning. When reading, Riak will wait for r

Re: Store whole database in memory

2011-05-28 Thread Michael McClain
Thank you, Mike and Greg, for the response. I've just replied to the list. In my use case, I need to be able to write 100,000 keys per second. Where the key is very small (12 bytes). And I always insert 1000 keys at once, in a bulk insert. I would also like to preserve the locality of the keys inse

Re: Store whole database in memory

2011-05-28 Thread Michael McClain
Thank you. That's what I'm looking for. I have stopped my 3 nodes (the ones from the Fast Track Tutorial) and set the storage to *riak_kv_ets_backend*. Then I started the 3 nodes again and tried some bulk insert operations with the java client (the one optimized for the api buffer). I can't see

Re: Store whole database in memory

2011-05-28 Thread Mike Oxford
With enough RAM you could just have it keep the whole thing in disk-cache... -mox On Fri, May 27, 2011 at 11:11 PM, Greg Nelson wrote: > Michael, > > You might want to check out riak_kv_ets_backend, riak_kv_gb_trees_backend, > and riak_kv_cache_backend. > > http://wiki.basho.com/Configuration-

Re: Store whole database in memory

2011-05-27 Thread Greg Nelson
Michael, You might want to check out riak_kv_ets_backend, riak_kv_gb_trees_backend, and riak_kv_cache_backend. http://wiki.basho.com/Configuration-Files.html -Greg On Friday, May 27, 2011 at 10:35 PM, Michael McClain wrote: > Hi, > > Is it possible to store the whole database in memory? > In

Store whole database in memory

2011-05-27 Thread Michael McClain
Hi, Is it possible to store the whole database in memory? In a similar way as Redis does. I'm really interested in the distributed map reduce done by riak ("bring processing to the data, instead of data to processors), but I need faster writes/reads that a memory-only database could provide.