Re: Incremental Data Processing With Hive UDAF

2013-01-17 Thread buddhika chamith
Hi All, Greatly appreciate any feedback on this. May be this may sound infeasible. Just wanted check with the experts on this. Anyway the problem of incremental data processing is a very interesting one if it can be accommodated for. Best Regards Buddhika On Wed, Jan 16, 2013 at 12:36 PM, buddhi

Re: Incremental Data Processing With Hive UDAF

2013-01-15 Thread buddhika chamith
Hi All, After digging in to the code more I realized that GroupbyOperator can be present at the map side of the computation as well, in which case it's doing partial computations. So in that case the terminate of UDAF will get called for partial results. However for the queries that I tried the te

Re: Incremental Data Processing With Hive UDAF

2013-01-14 Thread buddhika chamith
Any suggestions on this are greatly appreciated. Any one see major road blocks on this? Regards Buddhika On Sat, Jan 12, 2013 at 10:31 AM, buddhika chamith wrote: > Hi All, > > In order to achieve above I am researching on the feasibility of using a > set of custom UADFs for distributive aggrega