Riak doesn't use consistent hashing

I've been doing some digging through the details of how a node joins a cluster. When you hear that Riak uses consistent hashing, you'd expect it to distribute keys to nodes by hashing keys onto the ring AND hashing nodes onto the ring. Keys belong to the closest node on the ring, in the clockwis

Clarification on key filters

An MR job has 0 or more key filters. Adding a few transforms generates a lovely list of lists: { "inputs":{ "bucket":"invoices" "key_filters":[["tokenize", "-", 1], ["to_lower"], ["matches", "solutions"]] }, // ...} That makes sense: we've

Re: Issues with capacity planning pages on wiki

Hi, Anthony. There are really three different things below: 1- reducing the minimum overhead of the {Bucket, Key} encoding when riak is storing into bitcask 2- reducing the size of the vector clock encoding 3- reducing the size of the overall riak_object structure and metadata All three of the

Riak Recap for May 23 - 24

Afternoon, Evening, Morning to All - For today's Recap: tons of new Python code, a new Riak meetup on the calendar in San Francisco, a new README for merge_index, and more. Enjoy. Mark Community Manager Basho Technologies wiki.basho.com twitter.com/pharkmillups -

Re: Javascript VMs settings

Thanks Grant! --gordon On May 25, 2011, at 15:00 , Grant Schofield wrote: > Yes, that is perfectly safe. > > Grant > > On May 25, 2011, at 2:30 PM, Gordon Tillman wrote: > >> Howdy Folks, >> >> Assuming one has re-written all their M/R code and pre-commit hooks in >> Erlang, is it OK to set

Re: Javascript VMs settings

Yes, that is perfectly safe. Grant On May 25, 2011, at 2:30 PM, Gordon Tillman wrote: > Howdy Folks, > > Assuming one has re-written all their M/R code and pre-commit hooks in > Erlang, is it OK to set all these to 0 to save memory: > > map_js_vm_count > reduce_js_vm_count > hook_js_vm_count

Javascript VMs settings

Howdy Folks, Assuming one has re-written all their M/R code and pre-commit hooks in Erlang, is it OK to set all these to 0 to save memory: map_js_vm_count reduce_js_vm_count hook_js_vm_count Thanks! --gordon ___ riak-users mailing list riak-users@l

Re: Issues with capacity planning pages on wiki

Hi Nico, Thanks for the awesome analysis, the one part I was at first confused about was this part On Wed, May 25, 2011 at 06:22:53PM +0200, Nico Meyer wrote: > Your actual RAM consumption of 70 GB seems to be at odd with the > output of erlang:memory/0 that you sent: > > {total,7281790968} =>

Re: Issues with capacity planning pages on wiki

Nico - To echo Jonathan's sentiments from above, thanks for putting this together! This is an amazing write-up. I went ahead and opened a new issue to make sure we get this information incorporated into the wiki in a timely manner. https://github.com/basho/riak_wiki/issues/101

Re: Issues with capacity planning pages on wiki

Everybody feel free to steal from my mail to his/her heart's content :-). At the very least its now available in the mailing list archive for easy reference (http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-May/004292.html). I just hope my use of HTML formatting made it through

Re: Issues with capacity planning pages on wiki

That was one hell of a response. You need to post that as a Wiki article or such, after all that work :-O* Jonathan Langevin Systems Administrator Loom Inc. Wilmington, NC: (910) 241-0433 - jlange...@loomlearning.com - www.loomlearning.com - Skype: intel352 * On We

Re: Issues with capacity planning pages on wiki

Hi Anthony, I think, I can explain at least a big chunk of the difference in RAM and disk consumption you see. Let start with RAM. I could of course be wrong here, but I believe the /'static bitcask per key overhead/' is just plainly too small. Let me explain why. The bitcask_keydir_entry s

Re: General Memory/Performance Tuning Guidelines

Morning Rusty, Thanks very much for your time and trouble. Great info, very helpful, and very timely! Regards, --gordon On May 24, 2011, at 16:32 , Rusty Klophaus wrote: Hi Gordon, I have limited knowledge of configuring Innostore but can help answer some of your merge_index questions. T