Re: Hive 0.12 MAPJOIN hangs sometimes

2016-03-11 Thread Jason Dere
A join between bigint and string might actually be evaluated by converting both values to a double .. try doing an EXPLAIN of the query, that might show what conversion is being done for the join keys. If that is the case, you could try explicitly casting the join keys to either string or bigin

RE: Hive 0.12 MAPJOIN hangs sometimes

2016-03-11 Thread Yong Zhang
No, The join columns are ("bigint" and "string"). Yong Subject: RE: Hive 0.12 MAPJOIN hangs sometimes From: pjayachand...@hortonworks.com To: user@hive.apache.org; user@hive.apache.org Date: Fri, 11 Mar 2016 15:48:52 + Is the join column of type double? If so there is a known issue wit

Re: Hive StreamingAPI leaves table in not consistent state

2016-03-11 Thread Alan Gates
I believe this is an issue in the Storm Hive bolt. I don’t have an Apache JIRA on it, but if you ask on the Hortonworks lists we can connect you with the fix for the storm bolt. Alan. > On Mar 10, 2016, at 04:02, Igor Kuzmenko wrote: > > Hello, I'm using Hortonworks Data Platform 2.3.4 which

RE: Hive 0.12 MAPJOIN hangs sometimes

2016-03-11 Thread Prasanth Jayachandran
Is the join column of type double? If so there is a known issue with DoubleWritable hash collisions that makes hash join insanely slow. Thanks Prasanth On Fri, Mar 11, 2016 at 7:33 AM -0800, "Yong Zhang" mailto:java8...@hotmail.com>> wrote: I understand the Hive version problem. We are usi

RE: Hive 0.12 MAPJOIN hangs sometimes

2016-03-11 Thread Yong Zhang
I understand the Hive version problem. We are using IBM BigInsights V3.0.0.2, which comes with Hadoop 2.2.0 and Hive 0.12. It is extremely difficult to upgrade to BigInsights v4.x, as IBM did V4 totally different as V3. We are looking for the option to upgrade, but it won't be a fast way. The q

Re: Hive 0.12 MAPJOIN hangs sometimes

2016-03-11 Thread Jörn Franke
Honestly 0.12 is a no go - you miss a lot of performance improvements. Probably your query would execute in less than a minute. If your Hadoop vendor does not support smooth upgrades then change it. Hive 1.2.1 is the absolute minimum including using Orc or parquet as a table format and tez (pref

Hive 0.12 MAPJOIN hangs sometimes

2016-03-11 Thread Yong Zhang
Hi, Hive users: Currently our Hadoop vendor comes with Hive 0.12. I know it is a kind of old version, but upgrade still has some long path to go. Right now, we are facing an issue in the Hive 0.12. We have one ETL kind of steps implemented in Hive, and due to the data volume in this step, we know

Re: Tez job submissions failing when cluster is under provisioned..

2016-03-11 Thread Gautam
This one seems related https://issues.apache.org/jira/browse/YARN-4538 Yet to ascertain if it actually fixes this issue. On Thu, Mar 10, 2016 at 11:43 PM, Gopal Vijayaraghavan wrote: > > > This seems to be something YARN fair-scheduler reporting it this way.. > >although Tez doesn't seem to ha

Re: HBase table map to hive

2016-03-11 Thread Divya Gehlot
yes you can Register table in hive based in hbase CREATE EXTERNAL TABLE IF NOT EXISTS DB_NAME.TABLE_NAME(COL1 STRING,COL2 STRING,COL3 STRING) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:COL2,cf:COL3") TBLPROPERTIES ("hbase.t

HBase table map to hive

2016-03-11 Thread ram kumar
Hi, I have a HBase table with rowkey and column family. Is it possible to map HBase table to hive table? Thanks