Re: Query Optimization in Hive

2011-02-01 Thread bharath vissapragada
Hi , I updated the JIRA . Kindly give your suggestions so that I can go ahead and complete the task. Thanks On Tue, Feb 1, 2011 at 12:25 PM, bharath vissapragada wrote: > Thanks for replying namit.. > > It is motivating to receive a mail from the authors of Hive :). > > I filed the jira based

Re: Query Optimization in Hive

2011-01-31 Thread bharath vissapragada
Thanks for replying namit.. It is motivating to receive a mail from the authors of Hive :). I filed the jira based on the discussion.. https://issues.apache.org/jira/browse/HIVE-1938 I will try to update my idea asap. Thanks Bharath,V 4th year Undergrad,IIIT Hyderabad. w: http://research.iiit.a

Re: Query Optimization in Hive

2011-01-31 Thread Namit Jain
Bharath, This would be great. Why don¹t you write up something about how you are planning to proceed ? File a new jira and load some design notes/spec. there. We can definitely sync up. from there. This feature would be very useful to the community - We, at facebook, Would definitely like to us

Re: Query Optimization in Hive

2011-01-31 Thread bharath vissapragada
Hi Ning,Anja, I am doing my Masters thesis on this topic . I have implemented all SQL features like joins , selects etc on top of Hadoop (before knowing about Hive) and we have derived some basic cost-models for join re-ordering which seem to be working fine on some basic scales of TPCH datasets .

Re: Query Optimization in Hive

2011-01-31 Thread Ning Zhang
Hi Anja, As you noticed Hive only have limited supports for cost-baesd optimization. One of the reasons is that Hive used to have very small number of optional execution plans to choose from. One exception is mapjoin vs common joins. Liying Tang had some work on his last intern to convert commo

Re: Query Optimization in Hive

2011-01-31 Thread Ajo Fod
I think there is a developer mailing list ... that is probably the best place for this question. Also, I think there is a cost-based query optimizer in the works somewhere. -Ajo On Mon, Jan 31, 2011 at 2:04 PM, Anja Gruenheid wrote: > Hi! > > I'm a graduate student from Georgia Tech and I'm wor

Query Optimization in Hive

2011-01-31 Thread Anja Gruenheid
Hi! I'm a graduate student from Georgia Tech and I'm working with Hive for a research project. I am interested in query optimization and the Hive MetaStore in that context. Working through the documentation and code, I noticed that the implementation right now is using a rule-based optimizati