I will be loosening the key filter criterion after I get the basics working,
which I thought would be a simple equality check. 8M keys isn't really a
large data set, is it? I thought that keys were stored in memory and key
filters just operated on those memory keys and not data.
Jim
From: Ryan Caught <rcau...@gmail.com>
Date: Sun, 23 Oct 2011 14:52:48 -0400
To: Jim Adler <jim.ad...@comcast.net>
Cc: "riak-users@lists.basho.com" <riak-users@lists.basho.com>
Subject: Re: Key Filter Timeout
If you are doing just a simple equality check in the key filter, then why
not skip key filters and lookup the key directly? Key filters are not
performant over large data sets.
On Sun, Oct 23, 2011 at 2:38 PM, Jim Adler <jim.ad...@comcast.net> wrote:
> I'm trying to run a very simplified key filter that's timing out. I've got
> about 8M keys in a 3-node cluster, 15 GB memory, num_partitions=256, LevelDB
> backend.
>
> I'm thinking this should be pretty quick. What am I doing wrong?
>
> Jim
>
> Here's the query:
>
> curl -v -d
> '{"inputs":{"bucket":"nodes","key_filters":[["eq","user_id-xxxxxxx-info"]]},"q
> uery":[{"reduce":{"language":"erlang","module":"riak_kv_mapreduce","function":
> "reduce_identity"}}]}' -H "Content-Type: application/json"
> http://xx.xx.xx.xx:8098/mapred
>
> Here's the log:
>
> 18:25:08.892 [error] gen_fsm <0.20795.0> in state executing terminated with
> reason: {error,flow_timeout}
> 18:25:08.961 [error] CRASH REPORT Process <0.20795.0> with 2 neighbours
> crashed with reason: {error,flow_timeout}
> 18:25:08.963 [error] Supervisor luke_flow_sup had child undefined started with
> {luke_flow,start_link,undefined} at <0.20795.0> exit with reason
> {error,flow_timeout} in context child_terminated
> 18:25:08.966 [error] gen_fsm <0.20798.0> in state waiting_kl terminated with
> reason: {error,flow_timeout}
> 18:25:08.971 [error] CRASH REPORT Process <0.20798.0> with 0 neighbours
> crashed with reason: {error,flow_timeout}
> 18:25:08.980 [error] Supervisor riak_kv_keys_fsm_legacy_sup had child
> undefined started with {riak_kv_keys_fsm_legacy,start_link,undefined} at
> <0.20798.0> exit with reason {error,flow_timeout} in context child_terminated
> 18:25:08.983 [error] Supervisor luke_phase_sup had child undefined started
> with {luke_phase,start_link,undefined} at <0.20797.0> exit with reason
> {error,flow_timeout} in context child_terminated
> 18:25:08.996 [error] Supervisor luke_phase_sup had child undefined started
> with {luke_phase,start_link,undefined} at <0.20796.0> exit with reason
> {error,flow_timeout} in context child_terminated
>
>
>
> _______________________________________________
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
