Three cheers! Git clone && make all && make rel
- stat! On 2010-08-23, Justin Sheehy <jus...@basho.com> wrote: > Riak Users, > > One aspect of Riak's interface that has often been discouraged in the > past is the listing of all keys in a bucket. This has been for two > reasons: the first is that it is necessarily an operation that is more > heavyweight than any of the more targeted get/put/delete sorts of > things, but the second is that due to the priorities of the first many > users of Riak we hadn't really put much optimization into that area. > As a result, anything that required getting all keys from a bucket was > fairly slow and also fairly heavy in terms of memory consumption. > > We have put some effort into this recently and seen marked > improvement. The changes can be summed up as: > > 1- bitcask has a new fold_keys operation, which performs far less I/O > in most cases than the previous mechanism underlying list_keys. > > 2- the Riak backend interface to bitcask uses the new fold_keys operation. > > 3- the mechanism underlying the cluster-wide list_keys operation has > changed to require far less total memory in proportion to the list. > > Due to these three changes, there are two effective results: > > 1- In nearly all cases, the list_keys operator is much faster than > before. In some common cases it is 10 times faster. > > 2- In cases of very large buckets, memory allocation will not spike > during key listing. (though of course if you ask Riak to build the > whole list for you instead of streaming it out, then at least that > much must be used to accommodate) > > Note that since map/reduce uses the streaming list_keys under the hood > when performing map/reduce over a whole bucket, these changes affect > that interface's performance as well. > > The described changes are now in the trunks of the relevant > repositories, and will be included in the next release. > > -Justin > > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > -- Sent from my mobile device _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
