Re: Reading partition for reducer

2013-04-01 Thread Vikas Jadhav
yes i have joined u...@hadoop.apache.org mailing list. i think it is not possible to do in user code(map or reduce function) rather i dont want to make my changes visible to mapreduce programmer thats why i thought it is good idea to ask question here in this mailing list. On Tue, Apr 2, 2013 at 2:

Re: Reading partition for reducer

2013-04-01 Thread Steve Loughran
have you considered joining the u...@hadoop.apache.org and asking the question there? On 1 April 2013 17:38, Vikas Jadhav wrote: > Hi > > I want process/store all data pertaining to one reducer. > > i want store it in some data structure depending on key for example > > (0,ABC) > (0,TER) > (1,D

Re: Reading partition for reducer

2013-04-01 Thread Vikas Jadhav
Hi I want process/store all data pertaining to one reducer. i want store it in some data structure depending on key for example (0,ABC) (0,TER) (1,DEF) (1,XYZ) for key 0 and 1 data will be in different data structure. then perform cross product of above datasets Currently i am looking in Red

Re: Reading partition for reducer

2013-04-01 Thread Harsh J
The question should be more specific here: Do you want to process a map's sorted total output or do you want to pre-process a whole partition (i.e. all data pertaining to one reducer)? Former would be more ideal inside MapTask.java, latter in ReduceTask.java. On Mon, Apr 1, 2013 at 5:36 PM, Vikas