Re: Structure of Erlang MR Arg

2013-02-26 Thread Jeremiah Peschka
Excellent, thanks much for the clarification. --- Jeremiah Peschka - Founder, Brent Ozar Unlimited MCITP: SQL Server 2008, MVP Cloudera Certified Developer for Apache Hadoop On Tue, Feb 26, 2013 at 6:18 AM, Bryan Fink wrote: > On Feb 26, 2013, at 9:16 AM, Bryan Fink wrote: > >> {"reduce":{"la

Re: Structure of Erlang MR Arg

2013-02-26 Thread Bryan Fink
On Feb 26, 2013, at 9:16 AM, Bryan Fink wrote: >> {"reduce":{"language":"erlang", ... >> "arg":{"do_prereduce":true, >> "reduce_phase_batch_size": 250 ... I should have also mentioned that `do_prereduce` is actually a *map* phase argument. It is only applicable when a

Re: Structure of Erlang MR Arg

2013-02-26 Thread Bryan Fink
Doh … sorry this thread ended up lost in the stack. Response below… On Feb 14, 2013, at 10:58 AM, Jeremiah Peschka wrote: > In the MapReduce Implementation docs [1], it looks like the Arg parameter of > an Erlang MR phase is not a single argument but is, in fact, a proplist of > Erlang terms,