Hi Germain,
There is an issue in 0.12 that causes whole bucket map/reduce jobs to
ignore user supplied timeouts. This happens if the results are not
streamed. You can activate streaming by adding "chunked=true" to the
end of your request:
curl -v -XPOST http://localhost:8098/mapred?chunked=true -d
"{\"timeout\":600000,\"query\":[{\"map\":{\"source\":\"function(v) {
var data = Riak.mapValuesJson(v)[0] ; if ( data.faved_what !=
undefined && data.faved_what.total != undefined ) { return [
data.faved_what.total ] ; } return [];
}\",\"language\":\"javascript\",\"arg\":[],\"keep\":false}},{\"reduce\":{\"source\":\"function(values,
arg) { return [ values.reduce( function (item, total) { return item +
total*1 ; }, 0 ) ]
}\",\"language\":\"javascript\",\"arg\":[],\"keep\":true}}],\"inputs\":\"people\"}"
This issue has been fixed in 0.13:
https://issues.basho.com/show_bug.cgi?id=523
Thanks,
Dan
Daniel Reverri
Developer Advocate
Basho Technologies, Inc.
d...@basho.com <mailto:d...@basho.com>
On Tue, Oct 5, 2010 at 9:05 AM, Germain Maurice
<germain.maur...@linkfluence.net
<mailto:germain.maur...@linkfluence.net>> wrote:
Hello everybody,
I'm trying to execute some mapreduce jobs and i have a problem
with timeout.
I'm working with riak_kv_version : <<"0.12.1">> with a single node
"cluster".
My inputs are "people" bucket which contains more than 200k
documents. So i have decided to set mapreduce timeout to 600000
milliseconds.
Obviously, it doesn't take care of timeout.
You'll find below the JSON object i sent to Riak/mapred and the
Internal Server Error less than 60 seconds after. I added the
error message i got
in riak console.
=====
> date; perl riak.pl <http://riak.pl> ; date
mardi 5 octobre 2010, 17:47:58 (UTC+0200)
---
"{\"timeout\":600000,\"query\":[{\"map\":{\"source\":\"function(v)
{ var data = Riak.mapValuesJson(v)[0] ; if ( data.faved_what !=
undefined && data.faved_what.total != undefined ) { return [
data.faved_what.total ] ; } return [];
}\",\"language\":\"javascript\",\"arg\":[],\"keep\":false}},{\"reduce\":{\"source\":\"function(values,
arg) { return [ values.reduce( function (item, total) { return
item + total*1 ; }, 0 ) ]
}\",\"language\":\"javascript\",\"arg\":[],\"keep\":true}}],\"inputs\":\"people\"}"
--- 500 Internal Server Error
mardi 5 octobre 2010, 17:48:53 (UTC+0200)
=====
********
=ERROR REPORT==== 5-Oct-2010::17:48:53 ===
Failed reduce: {timeout,
{gen_server,call,
[<0.143.0>,
{dispatch,
{<0.143.0>,#Ref<0.0.5.43514>},
{{jsanon,
<<"function(values, arg) { return [ values.reduce( function (item,
total) { return item + total*1 ; }, 0 ) ] }">>},
[<<"2607">>,<<"261">>,<<"1753">>,<<"81">>,
<<"18190">>,<<"581">>,<<"811">>,<<"732">>,
<<"1831">>,<<"967">>,<<"2650">>,<<"222">>,
<<"674">>,<<"2136">>,<<"1467">>,<<"362">>,
<<"19965">>,<<"2025">>,<<"348">>,<<"4817">>,
<<"286">>],
[]}},
10000]}}
=ERROR REPORT==== 5-Oct-2010::17:48:53 ===
** State machine <0.24966.6> terminating
** Last event in was {inputs,[<<"286">>]}
** When State == executing
** Data == {state,1,riak_kv_reduce_phase,
{state,
{javascript,
{reduce,
{jsanon,
<<"function(values, arg) { return [ values.reduce( function (item,
total) { return item + total*1 ; }, 0 ) ] }">>},
[],true}},
[],[]},
true,true,undefined,[],undefined,1,<0.24964.6>,660000}
** Reason for termination =
** {error,failed_reduce}
********
Any idea about this ??
Thank you,
I'll go back to work on my slides for saturday :)
--
Germain Maurice
Administrateur Système/Réseau
Tel : +33.(0)1.42.43.54.33
http://www.linkfluence.net
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com <mailto:riak-users@lists.basho.com>
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
