Re: Incremental Data Processing With Hive UDAF

2013-01-17 Thread buddhika chamith
, buddhika chamith wrote: > Hi All, > > After digging in to the code more I realized that GroupbyOperator can be > present at the map side of the computation as well, in which case it's > doing partial computations. So in that case the terminate of UDAF will get > called for parti

Re: Incremental Data Processing With Hive UDAF

2013-01-15 Thread buddhika chamith
lity where aggregation function can be uniquely identified within the query? I realize this might be a long shot but I am still up for it if this is feasible albeit with some work. Or any other possible ways to achieve this is highly appreciated. Regards Buddhika On Mon, Jan 14, 2013 at 8:16 PM, buddhi

Re: Incremental Data Processing With Hive UDAF

2013-01-14 Thread buddhika chamith
Any suggestions on this are greatly appreciated. Any one see major road blocks on this? Regards Buddhika On Sat, Jan 12, 2013 at 10:31 AM, buddhika chamith wrote: > Hi All, > > In order to achieve above I am researching on the feasibility of using a > set of custom UADFs for

Incremental Data Processing With Hive UDAF

2013-01-11 Thread buddhika chamith
Hi All, In order to achieve above I am researching on the feasibility of using a set of custom UADFs for distributive aggregate operations (e.g: sum, count etc..). Idea is to incorporate some state persisted from earlier aggregations to the current aggregation value inside merge of the UDAF. For d

Re: Hive on Standalone Machine

2012-05-01 Thread buddhika chamith
ill building a command line it seems. I assumed you were > working to launch hive without a fork. Pretty cool though. > > On Tue, May 1, 2012 at 1:08 PM, buddhika chamith > wrote: > > Hi Ed, > > > > On Thu, Apr 26, 2012 at 12:26 PM, buddhika chamith <

Re: Hive on Standalone Machine

2012-05-01 Thread buddhika chamith
Hi Ed, On Thu, Apr 26, 2012 at 12:26 PM, buddhika chamith wrote: > > > On Thu, Apr 26, 2012 at 10:07 AM, Edward Capriolo > wrote: > >> we have an open ticket to run jobs without bin/hadoop do you care to >> share that code? >> > > Sure I will provide a pa

Re: Hive on Standalone Machine

2012-04-25 Thread buddhika chamith
On Thu, Apr 26, 2012 at 10:07 AM, Edward Capriolo wrote: > we have an open ticket to run jobs without bin/hadoop do you care to > share that code? > Sure I will provide a patch. Regards Buddhika > > On Wed, Apr 25, 2012 at 11:56 PM, buddhika chamith > wrote: > >

Re: Hive on Standalone Machine

2012-04-25 Thread buddhika chamith
Hi, If you are talking about running Hive in local mode without pointing to a $HADOOP_HOME I don't think it would work even if you include hadoop-core dependency in Hive classpath since in local mode Hive will use hadoop scripts present in $HADOOP_HOME/bin. I came across the same requirement rece

Re: Getting CNF during map reduce execution

2012-04-09 Thread buddhika chamith
rote: > >> HI Buddhika, >> >> Which version of Hive are you running? >> >> Thanks. >> >> Carl >> >> On Thu, Apr 5, 2012 at 10:12 AM, buddhika chamith < >> chamibuddh...@gmail.com> wrote: >> >>> Hi All, >>&

Re: Thrift Server not starting with Hive 0.8.1 and hadoop-1.0.1

2012-04-05 Thread buddhika chamith
Hi Shin, Not sure what do you mean by hangs here. I think this is about what the server would output on proper startup. (it is at least for me. :)) Did you try invoking hive sever using hive jdbc client [1] or equivalent? If not can you try and see whether it works. Regards Buddhika [1] https:/

Re: Duplicated Hadoop Package Structure in Hive exec

2012-04-02 Thread buddhika chamith
itself so getting them from distributed cache/auxjars > does not work well. I think the solution is we should probably > repackage commons util into hive-commons so their is not conflicts. > > Open up a jira for discussion. > > Edward > > On Mon, Apr 2, 2012 at 5:54 AM, budd

Re: non-equality joins

2012-03-16 Thread buddhika chamith
Hi, I think matt's solution is the way to go for now. If you need some basic understanding on how reduce and map side joins work see [1] whether if it helps you. Regards Buddhika [1] http://chamibuddhika.wordpress.com/2012/02/26/joins-with-map-reduce/ On Sat, Mar 17, 2012 at 6:41 AM, Alan Gates

Re: error running hive job

2012-02-12 Thread buddhika chamith
Hi Chung, >From my short experience on Hive this may happen due to various reasons. To find the real root cause you may need to dig in to hadoop tasktracker logs and see what's present there. Also have a look at the task log from the web UI (it gets printed to the console at the start of the Hive