On 23 Jun 2011, at 16:55, Jeremiah Peschka wrote:
> HTTP link walking will get you back the data in the way that you'd expect.
>
> It's a two-step process using PBC. MR link phases will give you a list of
> [bucket, key, tag] that you can then use to pull back the records from Riak.
>
The new Java client at https://github.com/basho/riak-java-client/ does a 2 step
link walk via map/reduce as a single API call (for the sake of API parity).
There is an example in the integration tests.
> ---
> Jeremiah Peschka
> Founder, Brent Ozar PLF, LLC
>
> On Thursday, June 23, 2011 at 8:52 AM, Andrew Berman wrote:
>
>> Ah, ok, that makes sense. One more question, when I use the HTTP link
>> walking, I do get the data back as expected, so is there a way to
>> replicate this in a Map-Reduce job or using the Erlang PBC (which I
>> forgot to mention is what I'm using and the reason I'm not using the
>> HTTP link walking method)?
>>
>> --Andrew
>>
>> On Thu, Jun 23, 2011 at 8:39 AM, Mathias Meyer <math...@basho.com> wrote:
>>> Andrew,
>>>
>>> the data looks like JSON, but it's not valid JSON. Have a look at the list
>>> that's in the data section (which is your BERT encoded data), the first
>>> character in that list is 131, which is not a valid UTF-8 character, and
>>> JSON only allows valid UTF-8 characters. With a binary-encoded format,
>>> there's always a chance for a control character like that to blow up the
>>> JSON generated before and after the MapReduce code is executed. With JSON,
>>> content agnosticism only goes as far as the set of legal characters allows.
>>>
>>> On a side note, if the data were a valid representation of a string, you
>>> would see it as a string in the log file as well, not just as a list of
>>> numbers.
>>>
>>> Mathias Meyer
>>> Developer Advocate, Basho Technologies
>>>
>>>
>>> On Donnerstag, 23. Juni 2011 at 17:31, Andrew Berman wrote:
>>>
>>>> But isn't the value itself JSON? Meaning this part:
>>>>
>>>> {struct,
>>>> [{<<"bucket">>,<<"user">>},
>>>> {<<"key">>,<<"LikiWUPJSFuxtrhCYpsPfg">>},
>>>> {<<"vclock">>,
>>>>
>>>> <<"a85hYGBgzGDKBVIsLKaZdzOYEhnzWBmes6Yd58sCAA==">>},
>>>> {<<"values">>,
>>>> [{struct,
>>>> [{<<"metadata">>,
>>>> {struct,
>>>> [{<<"X-Riak-VTag">>,
>>>> <<"1KnL9Dlma9Yg4eMhRuhwtx">>},
>>>> {<<"X-Riak-Last-Modified">>,
>>>> <<"Fri, 10 Jun 2011 03:05:11 GMT">>}]}},
>>>> {<<"data">>,
>>>>
>>>> <<131,108,0,0,0,18,104,2,100,0,6,114,...>>}]}]}
>>>>
>>>> So the only thing that is not JSON is the data itself, but when I get
>>>> the value, shouldn't I be getting the all the info above which is JSON
>>>> encoded?
>>>>
>>>> Thank you all for your help,
>>>>
>>>> Andrew
>>>>
>>>> On Thu, Jun 23, 2011 at 8:17 AM, Sean Cribbs <s...@basho.com
>>>> (mailto:s...@basho.com)> wrote:
>>>>> The object has to be JSON-encoded to be marshalled into the Javascript VM,
>>>>> and also on the way out if the Accept header indicates application/json.
>>>>> So
>>>>> you have two places where it needs to be encodable into JSON.
>>>>> On Thu, Jun 23, 2011 at 11:14 AM, Andrew Berman <rexx...@gmail.com
>>>>> (mailto:rexx...@gmail.com)> wrote:
>>>>>>
>>>>>> Mathias,
>>>>>>
>>>>>> I thought Riak was content agnostic when it came to the data being
>>>>>> stored? The map phase is not running Riak.mapValuesJson, so why is
>>>>>> the data itself going through the JSON parser? The JSON value
>>>>>> returned by v with all the info is valid and I see the struct atom in
>>>>>> there so mochijson2 can parse it properly, but I'm not clear why
>>>>>> mochijson2 would be coughing at the data part.
>>>>>>
>>>>>> --Andrew
>>>>>>
>>>>>> On Thu, Jun 23, 2011 at 5:32 AM, Mathias Meyer <math...@basho.com
>>>>>> (mailto:math...@basho.com)> wrote:
>>>>>>> Andrew,
>>>>>>>
>>>>>>> you're indeed hitting a JSON encoding problem here. BERT is binary data,
>>>>>>> and won't make the JSON parser happy when trying to deserialize it,
>>>>>>> before
>>>>>>> handing it into the map phase. You have two options here, and none of
>>>>>>> them
>>>>>>> will involve JavaScript as the MapReduce language.
>>>>>>>
>>>>>>> 1.) Use the Protobuff API, use Erlang functions to return the value or
>>>>>>> object (e.g. riak_mapreduce:map_object_value or
>>>>>>> riak_kv_mapreduce:map_identity), and then run MapReduce queries with the
>>>>>>> content type 'application/x-erlang-binary'. However, you're constrained
>>>>>>> by
>>>>>>> client libraries here, e.g. Ruby and Python don't support this content
>>>>>>> type
>>>>>>> for MapReduce on the Protobuffs interface yet, so you'd either implement
>>>>>>> something custom, or resort to a client that does, riak-erlang-client
>>>>>>> comes
>>>>>>> to mind, though it was proven to be possible using the Java client too,
>>>>>>> thanks to Russell. See [1] and [2]
>>>>>>>
>>>>>>> 2.) Convert the result from BERT into a JSON-parseable structure inside
>>>>>>> an Erlang map function, before it's returned to the client.
>>>>>>>
>>>>>>> The second approach certainly is less restrictive in terms of API usage,
>>>>>>> but certainly involves some overhead inside of the MapReduce request
>>>>>>> itself,
>>>>>>> but is less prone to encoding/decoding issues with JSON.
>>>>>>>
>>>>>>> Mathias Meyer
>>>>>>> Developer Advocate, Basho Technologies
>>>>>>>
>>>>>>> [1]
>>>>>>> http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-June/004447.html
>>>>>>> [2]
>>>>>>> http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-June/004485.html
>>>>>>>
>>>>>>> On Donnerstag, 23. Juni 2011 at 07:59, Andrew Berman wrote:
>>>>>>>
>>>>>>>> Hey Ryan,
>>>>>>>>
>>>>>>>> Here is the error from the sasl log. It looks like some sort of
>>>>>>>> encoding error. Any thoughts on how to fix this? I am storing the
>>>>>>>> data as BERT encoded binary and I set the content-type as
>>>>>>>> application/octet-stream.
>>>>>>>>
>>>>>>>> Thanks for your help!
>>>>>>>>
>>>>>>>> Andrew
>>>>>>>>
>>>>>>>> ERROR REPORT==== 9-Jun-2011::21:37:05 ===
>>>>>>>> ** Generic server <0.5996.21> terminating
>>>>>>>> ** Last message in was {batch_dispatch,
>>>>>>>> {map,
>>>>>>>> {jsanon,<<"function(value) {return [value];}">>},
>>>>>>>> [{struct,
>>>>>>>> [{<<"bucket">>,<<"user">>},
>>>>>>>> {<<"key">>,<<"LikiWUPJSFuxtrhCYpsPfg">>},
>>>>>>>> {<<"vclock">>,
>>>>>>>>
>>>>>>>> <<"a85hYGBgzGDKBVIsLKaZdzOYEhnzWBmes6Yd58sCAA==">>},
>>>>>>>> {<<"values">>,
>>>>>>>> [{struct,
>>>>>>>> [{<<"metadata">>,
>>>>>>>> {struct,
>>>>>>>> [{<<"X-Riak-VTag">>,
>>>>>>>> <<"1KnL9Dlma9Yg4eMhRuhwtx">>},
>>>>>>>> {<<"X-Riak-Last-Modified">>,
>>>>>>>> <<"Fri, 10 Jun 2011 03:05:11 GMT">>}]}},
>>>>>>>> {<<"data">>,
>>>>>>>>
>>>>>>>> <<131,108,0,0,0,18,104,2,100,0,6,114,...>>}]}]}]},
>>>>>>>> <<"user">>,none]}}
>>>>>>>> ** When Server state ==
>>>>>>>> {state,<0.143.0>,riak_kv_js_map,#Port<0.92614>,true}
>>>>>>>> ** Reason for termination ==
>>>>>>>> ** {function_clause,[{js_driver,eval_js,
>>>>>>>> [#Port<0.92614>,{error,bad_encoding},5000]},
>>>>>>>> {riak_kv_js_vm,invoke_js,2},
>>>>>>>> {riak_kv_js_vm,define_invoke_anon_js,3},
>>>>>>>> {riak_kv_js_vm,handle_call,3},
>>>>>>>> {gen_server,handle_msg,5},
>>>>>>>> {proc_lib,init_p_do_apply,3}]}
>>>>>>>>
>>>>>>>> =CRASH REPORT==== 9-Jun-2011::21:37:05 ===
>>>>>>>> crasher:
>>>>>>>> initial call: riak_kv_js_vm:init/1
>>>>>>>> pid: <0.5996.21>
>>>>>>>> registered_name: []
>>>>>>>> exception exit:
>>>>>>>>
>>>>>>>> {function_clause,[{js_driver,eval_js,[#Port<0.92614>,{error,bad_encoding},5000]},{riak_kv_js_vm,invoke_js,2},{riak_kv_js_vm,define_invoke_anon_js,3},{riak_kv_js_vm,handle_call,3},{gen_server,handle_msg,5},{proc_lib,init_p_do_apply,3}]}
>>>>>>>> in function gen_server:terminate/6
>>>>>>>> in call from proc_lib:init_p_do_apply/3
>>>>>>>> ancestors: [riak_kv_js_sup,riak_kv_sup,<0.128.0>]
>>>>>>>> messages: []
>>>>>>>> links: [<0.142.0>,<0.6009.21>]
>>>>>>>> dictionary: []
>>>>>>>> trap_exit: false
>>>>>>>> status: running
>>>>>>>> heap_size: 4181
>>>>>>>> stack_size: 24
>>>>>>>> reductions: 2586
>>>>>>>> neighbours:
>>>>>>>> neighbour:
>>>>>>>> [{pid,<0.6009.21>},{registered_name,[]},{initial_call,{riak_kv_mapper,init,[Argument__1]}},{current_function,{gen,do_call,4}},{ancestors,[riak_kv_mapper_sup,riak_kv_sup,<0.128.0>]},{messages,[]},{links,[<0.5996.21>,<12337.6227.21>,<0.162.0>]},{dictionary,[]},{trap_exit,false},{status,waiting},{heap_size,987},{stack_size,53},{reductions,1043}]
>>>>>>>> =SUPERVISOR REPORT==== 9-Jun-2011::21:37:05 ===
>>>>>>>> Supervisor: {local,riak_kv_js_sup}
>>>>>>>> Context: child_terminated
>>>>>>>> Reason:
>>>>>>>>
>>>>>>>> {function_clause,[{js_driver,eval_js,[#Port<0.92614>,{error,bad_encoding},5000]},{riak_kv_js_vm,invoke_js,2},{riak_kv_js_vm,define_invoke_anon_js,3},{riak_kv_js_vm,handle_call,3},{gen_server,handle_msg,5},{proc_lib,init_p_do_apply,3}]}
>>>>>>>> Offender:
>>>>>>>>
>>>>>>>> [{pid,<0.5996.21>},{name,undefined},{mfargs,{riak_kv_js_vm,start_link,undefined}},{restart_type,temporary},{shutdown,2000},{child_type,worker}]
>>>>>>>>
>>>>>>>> On Wed, Jun 22, 2011 at 6:10 PM, Ryan Zezeski <rzeze...@basho.com
>>>>>>>> (mailto:rzeze...@basho.com)
>>>>>>>> (mailto:rzeze...@basho.com)> wrote:
>>>>>>>>>
>>>>>>>>> Andrew,
>>>>>>>>> Maybe you could elaborate on the error? I tested this against master
>>>>>>>>> (commit below) just now with success.
>>>>>>>>> 2b1a474f836d962fa035f48c05452e22fc6c2193 Change dependency to allow
>>>>>>>>> for R14B03 as well as R14B02
>>>>>>>>> -Ryan
>>>>>>>>> On Wed, Jun 22, 2011 at 7:03 PM, Andrew Berman <rexx...@gmail.com
>>>>>>>>> (mailto:rexx...@gmail.com)
>>>>>>>>> (mailto:rexx...@gmail.com)> wrote:
>>>>>>>>>>
>>>>>>>>>> Hello,
>>>>>>>>>> I'm having issues link walking using the Map Reduce link function.
>>>>>>>>>> I am using HEAD from Git, so it's possible that's the issue, but
>>>>>>>>>> here is
>>>>>>>>>> what is happening.
>>>>>>>>>> I've got two buckets, user and user_email where user_email contains
>>>>>>>>>> a link to the user.
>>>>>>>>>> When I run this:
>>>>>>>>>> {
>>>>>>>>>> "inputs": [
>>>>>>>>>> [
>>>>>>>>>> "user_email",
>>>>>>>>>> "myem...@email.com (mailto:myem...@email.com)"
>>>>>>>>>> ]
>>>>>>>>>> ],
>>>>>>>>>> "query": [
>>>>>>>>>> {
>>>>>>>>>> "link": {
>>>>>>>>>> "bucket": "user",
>>>>>>>>>> "tag": "user"
>>>>>>>>>> }
>>>>>>>>>> }
>>>>>>>>>> ]
>>>>>>>>>> }
>>>>>>>>>> I only get [["user","LikiWUPJSFuxtrhCYpsPfg","user"]] returned. The
>>>>>>>>>> second I add a map function, even the simplest one (function(v) {
>>>>>>>>>> [v] } I
>>>>>>>>>> get a "map_reduce error":
>>>>>>>>>> {
>>>>>>>>>> "inputs": [
>>>>>>>>>> [
>>>>>>>>>> "user_email",
>>>>>>>>>> "myem...@email.com (mailto:myem...@email.com)"
>>>>>>>>>> ]
>>>>>>>>>> ],
>>>>>>>>>> "query": [
>>>>>>>>>> {
>>>>>>>>>> "link": {"bucket":"user", "tag":"user"}
>>>>>>>>>> }
>>>>>>>>>> ,{
>>>>>>>>>> "map": {
>>>>>>>>>> "language": "javascript",
>>>>>>>>>> "source": "function(v) { return[v]; }"
>>>>>>>>>> }
>>>>>>>>>> }
>>>>>>>>>> ]
>>>>>>>>>> }
>>>>>>>>>> Is this functionality broken? I am following what it says on the
>>>>>>>>>> Wiki for the MapRed version of link walking. When I use HTTP link
>>>>>>>>>> walking,
>>>>>>>>>> it works correctly.
>>>>>>>>>> Thanks,
>>>>>>>>>> Andrew
>>>>>>>>>> _______________________________________________
>>>>>>>>>> riak-users mailing list
>>>>>>>>>> riak-users@lists.basho.com (mailto:riak-users@lists.basho.com)
>>>>>>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> riak-users mailing list
>>>>>>>> riak-users@lists.basho.com (mailto:riak-users@lists.basho.com)
>>>>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>>>>
>>>>>> _______________________________________________
>>>>>> riak-users mailing list
>>>>>> riak-users@lists.basho.com (mailto:riak-users@lists.basho.com)
>>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Sean Cribbs <s...@basho.com (mailto:s...@basho.com)>
>>>>> Developer Advocate
>>>>> Basho Technologies, Inc.
>>>>> http://www.basho.com/
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users@lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
> _______________________________________________
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
