RE: Performance problems with Hive script

2012-01-24 Thread Butani, Harish
obs or long scripts of SQL. It's in alpha state; I am looking for users to work with. Regards, Harish. From: Igor Tatarinov [mailto:i...@decide.com] Sent: Monday, January 23, 2012 11:27 PM To: user@hive.apache.org Subject: Re: Performance problems with Hive script To compute moving aver

Re: Performance problems with Hive script

2012-01-23 Thread Igor Tatarinov
To compute moving averages, you should implement a custom reducer instead of doing a big join. That will work *much* faster. Also, Hive already has date_add(). Why did you have to implement your own? https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-DateFunction

Performance problems with Hive script

2012-01-23 Thread Benjamin Poserow
I wrote, separately, a Hadoop job to calculate running averages of about 2000 stock tickers over a 180 day period as well as a Hive script which performs equivalent functionality. I have been using Amazon Elastic MapReduce as my platform for running these jobs. I have been trying for a while to