Re: Map side aggregations

2012-05-23 Thread Ranjith
Thanks philip. Thanks, Ranjith On May 23, 2012, at 4:15 AM, Philip Tromans wrote: > Hi Ranjith, > > I haven't checked the code (so this might not be true), but I think that the > map side aggregation stuff uses it's own hash map within the map phase to do > the aggregation, instead of using

Re: Map side aggregations

2012-05-23 Thread Philip Tromans
Hi Ranjith, I haven't checked the code (so this might not be true), but I think that the map side aggregation stuff uses it's own hash map within the map phase to do the aggregation, instead of using a combiner, so you wouldn't expect to see any combine input records. Have a look for parameters li

Re: Map side aggregations

2012-05-22 Thread Ranjith
Thanks Matt. I am not performing a join so does that matter? What does this local task do? Thanks, Ranjith On May 22, 2012, at 8:17 PM, "Tucker, Matt" wrote: > Try setting hive.auto.convert.join to true. The CLI will have a local task > before it starts a map-reduce job on the cluster. > >

Re: Map side aggregations

2012-05-22 Thread Tucker, Matt
Try setting hive.auto.convert.join to true. The CLI will have a local task before it starts a map-reduce job on the cluster. Matt On May 22, 2012, at 8:43 PM, "Raghunath, Ranjith" mailto:ranjith.raghuna...@usaa.com>> wrote: I have the parameter hive.map.aggr set to true. However, when I loo