Hi all,
I think it would be convenient to be able to specify map and reduce
query phase operations as Erlang funs when using the Erlang protocol
buffer client and when interacting with Riak via its REST API. Such a
feature would be useful during development and for exploratory queries
when the data in Riak is stored using Erlang's native serialization
format (term_to_binary). The advantage over {M, F, A} is that you
don't have to get (new) code loaded into each node in your cluster to
execute an Erlang-based query.
Serializing funs for execution on remote nodes in Erlang is fraught
with subtlety -- certainly not all funs can be used in this fashion.
An alternative is to specify the source for a fun as a string and
evaluate it on a remote node to obtain a fun -- just as is done for
Javascript functions specified in the REST API.
As an experiment, I put together a patch for riak_kv that adds a new
FunTerm type for a Query:
%%<li> {strfun, Fun} : Fun is a string (list or binary)
%% containing the definition of an anonymous
%% Erlang function.</li>
With these changes (code link below), you can do the following using
the pb client:
Mapper = "fun(G, _X, _Y) ->
O = riak_object:get_value(G),
[proplists:get_value(blah, O)]
end.".
riakc_pb_socket:mapred(Pid, Input, [{map, {strfun, Mapper}, unset, true}]).
Along similar lines, the changes allow Erlang funs to be specified in
the REST API:
{"reduce":{"language":"erlang",
"source":"
fun(ValueList, _Arg) ->
[lists:foldl(fun erlang:'+'/2, 0, ValueList)]
end.",
"keep":true}}
If there is interest, I'd be willing to cleanup/enhance the patch for
inclusion. I wanted to get some feedback before spending any more
time on it -- perhaps there are other issues with this approach that
I'm not aware of or a more elegant solution...
BTW, the inspiration for the approach came from this post by the
Erjang developer:
http://www.javalimit.com/2010/05/passing-funs-to-other-erlang-nodes.html
The code is here: http://github.com/seth/riak_kv
+ seth
--
Seth Falcon | @sfalcon | http://userprimary.net/
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
