Re: single MR stage for join and group by

2013-08-02 Thread Chen Song
We are currently using cloudera's CDH4.2.0 based on Hive 0.10.0. So neither optimizations were incorporated. Thank you all for the valuable feedback and this is really helpful. I will look further into the details of the JIRA. On Fri, Aug 2, 2013 at 12:14 AM, Yin Huai wrote: > If the join

Re: single MR stage for join and group by

2013-08-01 Thread Yin Huai
If the join is a reduce side join, https://issues.apache.org/jira/browse/HIVE-2206 will optimize this query and generate a single MR job. The optimizer introduced by HIVE-2206 is in trunk. Currently, it only handles the same column(s). If the join is a MapJoin, hive 0.11 can generate a single MR j

Re: single MR stage for join and group by

2013-08-01 Thread Stephen Sprague
and what version of hive are you running your test on? i do believe - not certain - that hive 0.11 includes the optimization you seek. On Thu, Aug 1, 2013 at 10:19 AM, Chen Song wrote: > Suppose we have 2 simple tables > > A > id int > value string > > B > id > > When hive translates the follo