Re: riak with leveldb timeouts when retrieving bucket keys

[Kelly McLaughlin]

Artem, 

Are you using a riak package or have you built from source? Which version or 
branch are you using? Also do you know how many objects are in the bucket. 
Thanks.

Kelly

On Sep 23, 2011, at 2:23 AM, Artem Kozarezov wrote:

> Riak becomes unavailable when fetching bucket keys.
> wget "http://127.0.0.1:8098/riak/bucket?keys=stream";
> 
> 2011-09-23 12:13:21.924 [error] <0.1110.0> CRASH REPORT Process [] with 0 
> neighbours crashed with reason: 
> {timeout,{gen_server,call,[<0.1116.0>,{work,<0..0>,{fold,#Fun,#Fun},{fsm,{33509606,{1278813932664540053428224228626747642198940975104,'riak@188.138.84.236'}},<0.14780.226>}}]}}
> 2011-09-23 12:13:31.198 [error] <0.907.0> gen_fsm <0.907.0> in state ready 
> terminated with reason: 
> {timeout,{gen_server,call,[<0.916.0>,{work,<0.908.0>,{fold,#Fun,#Fun},{fsm,{33509606,{1187470080331358621040493926581979953470445191168,'riak@188.138.84.236'}},<0.14780.226>}}]}}
> 2011-09-23 12:14:22.196 [error] <0.1512.0> gen_fsm <0.1512.0> in state ready 
> terminated with reason: 
> {timeout,{gen_fsm,sync_send_all_state_event,[<0.1513.0>,stop]}}
> 
> Would be nice to have a configurable bandwidth limit for the keys retrieval 
> (e.g. "please give me no more than 1000 keys per second"), so that one could 
> fetch the keys without making Riak unresponsive.
> 
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

MR Output Changes

[Jeremiah Peschka]

While building out functionality in CorrugatedIron to coincide with the Riak 
1.0.0 release, I've run into something a bit odd.

MapReduce phases with a keep set to true are returning in multiple 
RpbMapRedResp messages. I expected this, especially since hte API says it's 
possible. What's causing the issue is the output format.

When I MR via the HTTP API, I get results like this:

[["riak_index_tests","5"],["riak_index_tests","6"],["riak_index_tests","7"],["riak_index_tests","4"],["riak_index_tests","8"],["riak_index_tests","3"],["riak_index_tests","2"],["riak_index_tests","0"],["riak_index_tests","1"],["riak_index_tests","9"]]

When I execute the same MR through the PB API I get results that look like this:

[["riak_index_tests","1"]][["riak_index_tests","6"]][["riak_index_tests","0"]][["riak_index_tests","7"]][["riak_index_tests","8"]][["riak_index_tests","4"]][["riak_index_tests","9"]][["riak_index_tests","2"]][["riak_index_tests","3"]][["riak_index_tests","5"]]

The MR query itself, just to make sure I haven't done something wrong, is this:

curl -H "Content-Type: application/json" $RIAK/mapred -d @- 
{"inputs":{"bucket":"riak_index_tests","index":"age_int","key":32},"query":[{"reduce":{"language":"erlang","module":"riak_kv_mapreduce","function":"reduce_identity","keep":true}}]}

and this:

{"inputs":{"bucket":"riak_index_tests","index":"age_int","key":32},"query":[{"reduce":{"language":"erlang","module":"riak_kv_mapreduce","function":"reduce_identity","keep":true}}]}

Is there a reason that Riak is returning discrete JSON chunks instead of 
streaming back raw bytes to the client? Is there less documented change in Riak 
1.0 that will prevent this type of behavior (which incidentally didn't crop up 
until the 1.0 pre-release builds).
---
Jeremiah Peschka - Founder, Brent Ozar PLF, LLC
Microsoft SQL Server MVP


___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com