RE: Hive 0.12 MAPJOIN hangs sometimes

2016-03-11 Thread Yong Zhang
ere is a known issue with DoubleWritable hash collisions that makes hash join insanely slow. Thanks Prasanth On Fri, Mar 11, 2016 at 7:33 AM -0800, "Yong Zhang" wrote: I understand the Hive version problem. We are using IBM BigInsights V3.0.0.2, which comes with Hadoo

RE: Hive 0.12 MAPJOIN hangs sometimes

2016-03-11 Thread Yong Zhang
er is configured wrongly that is why you may miss some messages. What is the exact join query. Hive on older version needed a special syntax if you wanted to benefit from partition pruning. Which Hadoop version are you using. On 11 Mar 2016, at 15:43, Yong Zhang wrote: Hi, Hive users: Currentl

Hive 0.12 MAPJOIN hangs sometimes

2016-03-11 Thread Yong Zhang
Hi, Hive users: Currently our Hadoop vendor comes with Hive 0.12. I know it is a kind of old version, but upgrade still has some long path to go. Right now, we are facing an issue in the Hive 0.12. We have one ETL kind of steps implemented in Hive, and due to the data volume in this step, we know