Counting the number of items in a bucket
I have a bucket which contains images refernced by a key which is made up from a guid. I have a number of servers in my cluster and my bucket is set up to store 3 versions of every item in the bucket across the servers in the cluster. I'd like to understand a little bit more about how my cluster is performing in terms of data storage. I can find out how much space each bitcask is currently taking up on each server, but I'd like to compare reference this against the number of unique keys that are being stored in the system. I've tried to use map-reduce methods using the erlang methods defined in the riak_kv_mapreduce, calling them against the HTTP interface but my queries are timing out. I've tried to extend the timeout, but it's still timing out. What's the most efficient way to find out how many keys exist in a particular bucket? ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Counting the number of items in a bucket
Thanks for that. I've tried to run this across my cluster but it just
returns an error message. Do I need to include a key filter as well?
An error occurred parsing the "inputs" field.
Key filter expression missing filter list.
On 22 November 2011 08:42, Fyodor Yarochkin wrote:
> You can try to limit input by particular subset of ranges, if that
> helps. Here's my count m-r snippet:
>
> {
> "inputs":{
> "bucket":"data",
> "index":"numero_int",
> "start":0,
> "end":1212
> },
> "query":[
>{"map":
> {"language":"erlang","module":"riak_kv_mapreduce","function":"map_object_value","arg":"filter_notfound"}
>},
> {"reduce": {"language":"erlang",
>
> "module":"riak_kv_mapreduce","function":"reduce_count_inputs","arg":"filter_notfound"}}
>
>], "timeout": 9000
> }
>
> hope it helps,
> -fyodor
>
>
>
> On Tue, Nov 22, 2011 at 3:26 AM, Stephen Bennett
> wrote:
> > I have a bucket which contains images refernced by a key which is made up
> > from a guid. I have a number of servers in my cluster and my bucket is
> set
> > up to store 3 versions of every item in the bucket across the servers in
> the
> > cluster. I'd like to understand a little bit more about how my cluster is
> > performing in terms of data storage. I can find out how much space each
> > bitcask is currently taking up on each server, but I'd like to compare
> > reference this against the number of unique keys that are being stored in
> > the system.
> >
> > I've tried to use map-reduce methods using the erlang methods defined in
> the
> > riak_kv_mapreduce, calling them against the HTTP interface but my queries
> > are timing out. I've tried to extend the timeout, but it's still timing
> out.
> >
> > What's the most efficient way to find out how many keys exist in a
> > particular bucket?
> > ___
> > riak-users mailing list
> > riak-users@lists.basho.com
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> >
> >
>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Moving Riak bitcask directory
I want to move my Riak bitcask directory onto a different filesystem partition in order to make use of more space that is available. Is it as simple as: 1. Stopping Riak 2. Moving the directory to the new partition 3. Sym-linking the directory to the old location 4. Starting Riak Is there a better way to do this, and is there anything that I should be looking out for when doing this? ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Counting the number of items in a bucket
Thanks.
This is an extremely efficient way of counting the size of the bucket.
On 22 November 2011 15:23, francisco treacy wrote:
> I would probably stream keys to the client and count them there (it's the
> most efficient method I can think of)
>
> If you have node.js installed, do this:
>
> npm install riak-js@latest
> node -e "require('riak-js').getClient({ port: 8098 }).count('bucket');"
>
>
> 2011/11/21 Stephen Bennett
>
>> I have a bucket which contains images refernced by a key which is made up
>> from a guid. I have a number of servers in my cluster and my bucket is set
>> up to store 3 versions of every item in the bucket across the servers in
>> the cluster. I'd like to understand a little bit more about how my cluster
>> is performing in terms of data storage. I can find out how much space each
>> bitcask is currently taking up on each server, but I'd like to compare
>> reference this against the number of unique keys that are being stored in
>> the system.
>>
>> I've tried to use map-reduce methods using the erlang methods defined in
>> the riak_kv_mapreduce, calling them against the HTTP interface but my
>> queries are timing out. I've tried to extend the timeout, but it's still
>> timing out.
>>
>> What's the most efficient way to find out how many keys exist in a
>> particular bucket?
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
