Re: how to specify MultipleOutputs, MultipleInputs in using Avro mapred API

2010-08-18 Thread Doug Cutting
On 08/18/2010 10:49 AM, Harsh J wrote: We hope to add more such tools for such conversion/ingest, e.g.: https://issues.apache.org/jira/browse/AVRO-458 Offtopic, but is there any work being done on this already? I saw one of them tagged with 'GSOC', so wish to know before I sink something down.

Re: how to specify MultipleOutputs, MultipleInputs in using Avro mapred API

2010-08-18 Thread Harsh J
On Wed, Aug 18, 2010 at 11:07 PM, Doug Cutting wrote: > On 08/18/2010 10:18 AM, ey-chih chow wrote: >> >> Thanks. But by doing this way, what kind of advantage we can get from >> Avro? > > The Avro MapReduce API is easiest to use when both inputs and outputs are > Avro data. > > If inputs are not

Re: how to specify MultipleOutputs, MultipleInputs in using Avro mapred API

2010-08-18 Thread Harsh J
from HBase and the log file >> > respectively? >> >  Also how can I configure an Avro reducer to generate multiple outputs? >> >  For >> > multiple inputs and outputs, I got some examples programs from Tom >> > White's >> > Hadoop book.  But

Re: how to specify MultipleOutputs, MultipleInputs in using Avro mapred API

2010-08-18 Thread Doug Cutting
On 08/18/2010 10:18 AM, ey-chih chow wrote: Thanks. But by doing this way, what kind of advantage we can get from Avro? The Avro MapReduce API is easiest to use when both inputs and outputs are Avro data. If inputs are not Avro data, but you want to use the rest of the Avro MR API, then you

RE: how to specify MultipleOutputs, MultipleInputs in using Avro mapred API

2010-08-18 Thread ey-chih chow
Thanks. But by doing this way, what kind of advantage we can get from Avro? Ey-Chih > From: qwertyman...@gmail.com > Date: Wed, 18 Aug 2010 19:39:17 +0530 > Subject: Re: how to specify MultipleOutputs, MultipleInputs in using Avro > mapred API > To: user@avro.apache.org >

Re: how to specify MultipleOutputs, MultipleInputs in using Avro mapred API

2010-08-18 Thread Harsh J
e inputs and outputs, I got some examples programs from Tom White's > Hadoop book.  But I simply don't know what kind of changes I should make for > the Avro case. > Ey-Chih > > ________________ > From: eyc...@hotmail.com > To: user@avro.apache.org > Subje

RE: how to specify MultipleOutputs, MultipleInputs in using Avro mapred API

2010-08-18 Thread ey-chih chow
from Tom White's Hadoop book. But I simply don't know what kind of changes I should make for the Avro case. Ey-Chih From: eyc...@hotmail.com To: user@avro.apache.org Subject: how to specify MultipleOutputs, MultipleInputs in using Avro mapred API Date: Mon, 16 Aug 2010 18:2

how to specify MultipleOutputs, MultipleInputs in using Avro mapred API

2010-08-16 Thread ey-chih chow
Hi, I got a Map/Reduce job that require multiple inputs and outputs. One of the inputs will be processed by a mapper and a reducer that are subclasses of AvroMapper/AvroReducer respectively. And the reducer has multiple outputs. I appreciate if anybody could let me know how to configure the

how to specify MultipleOutputs, MultipleInputs in using Avro mapred API

2010-08-16 Thread ey-chih chow
Hi, I got a Map/Reduce job that require multiple inputs and outputs. One of the inputs will be processed by a mapper and a reducer that are subclasses of AvroMapper/AvroReducer respectively. And the reducer has multiple outputs. I appreciate if anybody could let me know how to config