Mathias, I thought Riak was content agnostic when it came to the data being stored? The map phase is not running Riak.mapValuesJson, so why is the data itself going through the JSON parser? The JSON value returned by v with all the info is valid and I see the struct atom in there so mochijson2 can parse it properly, but I'm not clear why mochijson2 would be coughing at the data part.
--Andrew On Thu, Jun 23, 2011 at 5:32 AM, Mathias Meyer <math...@basho.com> wrote: > Andrew, > > you're indeed hitting a JSON encoding problem here. BERT is binary data, and > won't make the JSON parser happy when trying to deserialize it, before > handing it into the map phase. You have two options here, and none of them > will involve JavaScript as the MapReduce language. > > 1.) Use the Protobuff API, use Erlang functions to return the value or object > (e.g. riak_mapreduce:map_object_value or riak_kv_mapreduce:map_identity), and > then run MapReduce queries with the content type > 'application/x-erlang-binary'. However, you're constrained by client > libraries here, e.g. Ruby and Python don't support this content type for > MapReduce on the Protobuffs interface yet, so you'd either implement > something custom, or resort to a client that does, riak-erlang-client comes > to mind, though it was proven to be possible using the Java client too, > thanks to Russell. See [1] and [2] > > 2.) Convert the result from BERT into a JSON-parseable structure inside an > Erlang map function, before it's returned to the client. > > The second approach certainly is less restrictive in terms of API usage, but > certainly involves some overhead inside of the MapReduce request itself, but > is less prone to encoding/decoding issues with JSON. > > Mathias Meyer > Developer Advocate, Basho Technologies > > [1] > http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-June/004447.html > [2] > http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-June/004485.html > > On Donnerstag, 23. Juni 2011 at 07:59, Andrew Berman wrote: > >> Hey Ryan, >> >> Here is the error from the sasl log. It looks like some sort of >> encoding error. Any thoughts on how to fix this? I am storing the >> data as BERT encoded binary and I set the content-type as >> application/octet-stream. >> >> Thanks for your help! >> >> Andrew >> >> ERROR REPORT==== 9-Jun-2011::21:37:05 === >> ** Generic server <0.5996.21> terminating >> ** Last message in was {batch_dispatch, >> {map, >> {jsanon,<<"function(value) {return [value];}">>}, >> [{struct, >> [{<<"bucket">>,<<"user">>}, >> {<<"key">>,<<"LikiWUPJSFuxtrhCYpsPfg">>}, >> {<<"vclock">>, >> >> <<"a85hYGBgzGDKBVIsLKaZdzOYEhnzWBmes6Yd58sCAA==">>}, >> {<<"values">>, >> [{struct, >> [{<<"metadata">>, >> {struct, >> [{<<"X-Riak-VTag">>, >> <<"1KnL9Dlma9Yg4eMhRuhwtx">>}, >> {<<"X-Riak-Last-Modified">>, >> <<"Fri, 10 Jun 2011 03:05:11 GMT">>}]}}, >> {<<"data">>, >> >> <<131,108,0,0,0,18,104,2,100,0,6,114,...>>}]}]}]}, >> <<"user">>,none]}} >> ** When Server state == {state,<0.143.0>,riak_kv_js_map,#Port<0.92614>,true} >> ** Reason for termination == >> ** {function_clause,[{js_driver,eval_js, >> [#Port<0.92614>,{error,bad_encoding},5000]}, >> {riak_kv_js_vm,invoke_js,2}, >> {riak_kv_js_vm,define_invoke_anon_js,3}, >> {riak_kv_js_vm,handle_call,3}, >> {gen_server,handle_msg,5}, >> {proc_lib,init_p_do_apply,3}]} >> >> =CRASH REPORT==== 9-Jun-2011::21:37:05 === >> crasher: >> initial call: riak_kv_js_vm:init/1 >> pid: <0.5996.21> >> registered_name: [] >> exception exit: >> {function_clause,[{js_driver,eval_js,[#Port<0.92614>,{error,bad_encoding},5000]},{riak_kv_js_vm,invoke_js,2},{riak_kv_js_vm,define_invoke_anon_js,3},{riak_kv_js_vm,handle_call,3},{gen_server,handle_msg,5},{proc_lib,init_p_do_apply,3}]} >> in function gen_server:terminate/6 >> in call from proc_lib:init_p_do_apply/3 >> ancestors: [riak_kv_js_sup,riak_kv_sup,<0.128.0>] >> messages: [] >> links: [<0.142.0>,<0.6009.21>] >> dictionary: [] >> trap_exit: false >> status: running >> heap_size: 4181 >> stack_size: 24 >> reductions: 2586 >> neighbours: >> neighbour: >> [{pid,<0.6009.21>},{registered_name,[]},{initial_call,{riak_kv_mapper,init,[Argument__1]}},{current_function,{gen,do_call,4}},{ancestors,[riak_kv_mapper_sup,riak_kv_sup,<0.128.0>]},{messages,[]},{links,[<0.5996.21>,<12337.6227.21>,<0.162.0>]},{dictionary,[]},{trap_exit,false},{status,waiting},{heap_size,987},{stack_size,53},{reductions,1043}] >> =SUPERVISOR REPORT==== 9-Jun-2011::21:37:05 === >> Supervisor: {local,riak_kv_js_sup} >> Context: child_terminated >> Reason: >> {function_clause,[{js_driver,eval_js,[#Port<0.92614>,{error,bad_encoding},5000]},{riak_kv_js_vm,invoke_js,2},{riak_kv_js_vm,define_invoke_anon_js,3},{riak_kv_js_vm,handle_call,3},{gen_server,handle_msg,5},{proc_lib,init_p_do_apply,3}]} >> Offender: >> [{pid,<0.5996.21>},{name,undefined},{mfargs,{riak_kv_js_vm,start_link,undefined}},{restart_type,temporary},{shutdown,2000},{child_type,worker}] >> >> On Wed, Jun 22, 2011 at 6:10 PM, Ryan Zezeski <rzeze...@basho.com >> (mailto:rzeze...@basho.com)> wrote: >> > >> > Andrew, >> > Maybe you could elaborate on the error? I tested this against master >> > (commit below) just now with success. >> > 2b1a474f836d962fa035f48c05452e22fc6c2193 Change dependency to allow for >> > R14B03 as well as R14B02 >> > -Ryan >> > On Wed, Jun 22, 2011 at 7:03 PM, Andrew Berman <rexx...@gmail.com >> > (mailto:rexx...@gmail.com)> wrote: >> > > >> > > Hello, >> > > I'm having issues link walking using the Map Reduce link function. I am >> > > using HEAD from Git, so it's possible that's the issue, but here is what >> > > is happening. >> > > I've got two buckets, user and user_email where user_email contains a >> > > link to the user. >> > > When I run this: >> > > { >> > > "inputs": [ >> > > [ >> > > "user_email", >> > > "myem...@email.com (mailto:myem...@email.com)" >> > > ] >> > > ], >> > > "query": [ >> > > { >> > > "link": { >> > > "bucket": "user", >> > > "tag": "user" >> > > } >> > > } >> > > ] >> > > } >> > > I only get [["user","LikiWUPJSFuxtrhCYpsPfg","user"]] returned. The >> > > second I add a map function, even the simplest one (function(v) { [v] } >> > > I get a "map_reduce error": >> > > { >> > > "inputs": [ >> > > [ >> > > "user_email", >> > > "myem...@email.com (mailto:myem...@email.com)" >> > > ] >> > > ], >> > > "query": [ >> > > { >> > > "link": {"bucket":"user", "tag":"user"} >> > > } >> > > ,{ >> > > "map": { >> > > "language": "javascript", >> > > "source": "function(v) { return[v]; }" >> > > } >> > > } >> > > ] >> > > } >> > > Is this functionality broken? I am following what it says on the Wiki >> > > for the MapRed version of link walking. When I use HTTP link walking, it >> > > works correctly. >> > > Thanks, >> > > Andrew >> > > _______________________________________________ >> > > riak-users mailing list >> > > riak-users@lists.basho.com (mailto:riak-users@lists.basho.com) >> > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> _______________________________________________ >> riak-users mailing list >> riak-users@lists.basho.com (mailto:riak-users@lists.basho.com) >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
