Hi Vijay Your current error looks like some issue with the Select query. Is the select query working as desired? hive> SELECT * FROM pokes where foo=98;
Regards Bejoy KS ________________________________ From: vijay shinde <vijaysanj...@gmail.com> To: user@hive.apache.org; Bejoy Ks <bejoy...@yahoo.com> Sent: Friday, July 27, 2012 12:24 AM Subject: Re: HBASE and HIVE Integration Hi Bejoy, Thanks for your reply. My Hbase is working fine independently. I was able reolve the zookeeper error. I had issues with hive-site.xml configuration.. Now my job run little bit longer and this is the lastest error I am on Cloudera CDH3U4 demo VM Number of reduce tasks is set to 0 since there's no reduce operator Starting Job = job_201207260911_0003, Tracking URL = http://0.0.0.0:50030/jobdetails.jsp?jobid=job_201207260911_0003 Kill Command = /usr/lib/hadoop-0.20/bin/hadoop job -Dmapred.job.tracker=0.0.0.0:8021 -kill job_201207260911_0003 2012-07-26 13:34:55,900 Stage-0 map = 0%, reduce = 0% 2012-07-26 13:35:56,207 Stage-0 map = 0%, reduce = 0% 2012-07-26 13:36:56,364 Stage-0 map = 0%, reduce = 0% 2012-07-26 13:37:57,342 Stage-0 map = 0%, reduce = 0% 2012-07-26 13:38:57,398 Stage-0 map = 0%, reduce = 0% 2012-07-26 13:39:57,467 Stage-0 map = 0%, reduce = 0% 2012-07-26 13:40:57,852 Stage-0 map = 0%, reduce = 0% 2012-07-26 13:41:58,724 Stage-0 map = 0%, reduce = 0% 2012-07-26 13:42:59,700 Stage-0 map = 0%, reduce = 0% 2012-07-26 13:44:00,582 Stage-0 map = 0%, reduce = 0% 2012-07-26 13:45:00,977 Stage-0 map = 0%, reduce = 0% 2012-07-26 13:46:01,110 Stage-0 map = 0%, reduce = 0% 2012-07-26 13:46:05,443 Stage-0 map = 100%, reduce = 100% Ended Job = job_201207260911_0003 with errors hive> INSERT OVERWRITE TABLE hive_hbasetable_k SELECT * FROM pokes where foo=98; java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"foo":98,"bar":"val_98"} at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:161) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:270) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177) at org.apache.hadoop.mapred.Child.main(Child.java:264) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"foo":98,"bar":"val_98"} at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:546) at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143) ... 8 more Caused by: or FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask hive> On Thu, Jul 26, 2012 at 9:55 AM, Bejoy Ks <bejoy...@yahoo.com> wrote: Hi Vijay > >Is your hbase working independently without any issues. I mean, are you able >to insert data into hbase tables without using hive integration? > >Was the same error message thrown when you directly provided hbase.master >instead of zookeeper quorum? > >Regards >Bejoy KS > > >________________________________ >From: Hezhiqiang (Ransom) <ransom.hezhiqi...@huawei.com> >To: "user@hive.apache.org" <user@hive.apache.org> >Cc: Zhaojun (Terry) <terry.zhao...@huawei.com> >Sent: Thursday, July 26, 2012 9:06 AM >Subject: RE: HBASE and HIVE Integration > > > > >Hi，Vijay > You need to add zookeeper.jar to the hive-site.xml ><name>hive.aux.jars.path</name> ><value>file:///usr/lib/hive/lib/hive-hbase-handler-0.7.1-cdh3u4.jar,file:///usr/lib/hive/lib/hbase-0.92.jar,file:///usr/lib/hive/lib/zookeeper-3.4.3.jar,file:///usr/lib/hive/lib/hive-contrib-0.7.1-cdh3u4.jar, > file:///usr/lib/hive/lib/ zookeeper-3.4.3.jar</value> ></property> ><name>hbase.zookeeper.quorum</name> ><value>localhost</value> ></property> > > >Regards >Ransom > >From:vijay shinde [mailto:vijaysanj...@gmail.com] >Sent: Thursday, July 26, 2012 8:59 AM >To: user@hive.apache.org; Bejoy Ks >Subject: Re: HBASE and HIVE Integration > >Hi Bejoy, > >I made some changes as per your suggetion. > >Here is the error from the >http://0.0.0.0:50030/jobdetails.jsp?jobid=job_201207251858_0004 Job: > >Error: java.lang.ClassNotFoundException: org.apache.zookeeper.KeeperException > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > at java.lang.ClassLoader.loadClass(ClassLoader.java:307) > at sun.misc.Launcher$ > >I went ahead and updated the hadoop-env.sh file and set the class path for >hbase and zookeeper as follows: ># Extra Java CLASSPATH elements. Optional. >export >HADOOP_CLASSPATH="/usr/lib/hive/lib/hbase-0.92.jar:/usr/lib/hive/lib/zookeeper-3.4.3.jar:$HADOOP_CLASSPATH" > >Here is snippet of hive-site.xml file > ><name>hive.aux.jars.path</name> ><value>file:///usr/lib/hive/lib/hive-hbase-handler-0.7.1-cdh3u4.jar,file:///usr/lib/hive/lib/hbase-0.92.jar,file:///usr/lib/hive/lib/zookeeper-3.4.3.jar,file:///usr/lib/hive/lib/hive-contrib-0.7.1-cdh3u4.jar</value> ></property> ><name>hbase.zookeeper.quorum</name> ><value>localhost</value> ></property> > >Error message while executing hive query > >[root@localhost hive]# ./bin/hive >Hive history file=/tmp/root/hive_job_log_root_201207252044_1993919630.txt >hive> INSERT OVERWRITE TABLE hive_hbasetable_k SELECT * FROM pokes where >foo=98; >Total MapReduce jobs = 1 >Launching Job 1 out of 1 >Number of reduce tasks is set to 0 since there's no reduce operator >Starting Job = job_201207251858_0004, Tracking URL = >http://0.0.0.0:50030/jobdetails.jsp?jobid=job_201207251858_0004 >Kill Command = /usr/lib/hadoop-0.20/bin/hadoop job >-Dmapred.job.tracker=0.0.0.0:8021 -kill job_201207251858_0004 >2012-07-25 20:46:38,207 Stage-0 map = 0%, reduce = 0% >2012-07-25 20:47:35,920 Stage-0 map = 100%, reduce = 100% >Ended Job = job_201207251858_0004 with errors >FAILED: Execution Error, return code 2 from >org.apache.hadoop.hive.ql.exec.MapRedTask >hive> > >I am lost need help badly!!!!!!!!!!!!! > >Vijay > > > >On Wed, Jul 25, 2012 at 9:47 AM, Bejoy Ks <bejoy...@yahoo.com> wrote: >Hi Vijay > >You have provided the hbase master directly. (It is fine for single node >hbase installation). But still can you try providing the zookeeper quorum >instead. > >If that doesn't work as well , please post the error log from the mapreduce >tasks? > >Just go the jobtracker page and drill down on the corresponding job to get the >failed tasks. From each failed tasks you can get the error logs. >http://0.0.0.0:50030/jobdetails.jsp?jobid=job_201207250246_0005 > > >Regards >Bejoy KS > > >________________________________ > >From:vijay shinde <vijaysanj...@gmail.com> >To:user@hive.apache.org; bejoy...@yahoo.com >Sent: Wednesday, July 25, 2012 6:58 PM >Subject: Re: HBASE and HIVE Integration > >Hi Bejoy, > >Thanks for quick reply. Here are some additional details > >Cloudera Version - CDH3U4 > >hive-site.xml > > ><property> ><name>hive.aux.jars.path</name> ><value>file:///usr/lib/hive/lib/hive-hbase-handler-0.7.1-cdh3u2.jar,file:///usr/lib/hive/lib/hbase-0.90.4-cdh3u2.jar,file:///usr/lib/hive/lib/zookeeper-3.3.1.jar,file:///usr/lib/hive/lib/hive-contrib-0.7.1-cdh3u2.jar</value> ></property> >Execution Log > >1. start zookeeper >[root@localhost zookeeper]# ./bin/zkServer.sh start > >2. start hbase > >3. start hive. I am setting hive jars in hive-site.xml > >./bin/hive -hiveconf hbase.master=127.0.1.1:60010 > >4. Create new HBase table which is to be managed by Hive > >CREATE TABLE hive_hbasetable_k(key int, value string) >STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' >WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") >TBLPROPERTIES ("hbase.table.name" = "hivehbasek"); > >5. Create a logical table pokes in Hive >CREATE TABLE pokes (foo INT, bar STRING); >6. HIve error while inserting the data from Hive Poke table to HBASE table >hive> INSERT OVERWRITE TABLE hive_hbasetable_k SELECT * FROM pokes WHERE >foo=98; >Total MapReduce jobs = 1 >Launching Job 1 out of 1 >Number of reduce tasks is set to 0 since there's no reduce operator >Starting Job = job_201207250246_0005, Tracking URL = >http://0.0.0.0:50030/jobdetails.jsp?jobid=job_201207250246_0005 >Kill Command = /usr/lib/hadoop-0.20/bin/hadoop job >-Dmapred.job.tracker=0.0.0.0:8021 -kill job_201207250246_0005 >2012-07-25 04:26:00,198 Stage-0 map = 0%, reduce = 0% >2012-07-25 04:27:00,767 Stage-0 map = 0%, reduce = 0% >2012-07-25 04:27:08,844 Stage-0 map = 100%, reduce = 100% >Ended Job = job_201207250246_0005 with errors > >FAILED: Execution Error, return code 2 from >org.apache.hadoop.hive.ql.exec.MapRedTask >Let me know if you need any additonal information. > >Thanks, >Vijay >On Wed, Jul 25, 2012 at 5:30 AM, Bejoy KS <bejoy...@yahoo.com> wrote: >Hi Vijay > >Can you share more details like > >The CDH Version/Hive version you are using > >Steps you followed for hive hbase integration with the values you set > >The DDL used for hive hbase integration > >The actual error from failed map reduce task >Regards >Bejoy KS > >Sent from handheld, please excuse typos. > >________________________________ > >From: vijay shinde <vijaysanj...@gmail.com> >Date: Wed, 25 Jul 2012 04:45:41 -0400 >To: <user@hive.apache.org> >ReplyTo: user@hive.apache.org >Subject: HBASE and HIVE Integration > >I am facing issue while executing HIVE queries on HBASE-HIVE integration. >I followed the wiki hbase-hive integration >https://cwiki.apache.org/Hive/hbaseintegration.html > >I have already passed all the required jars for auxpath in hive-site.xml file. >I am using Cloudera CDH demo VM.. Any help would be highly appreciated > >hive> INSERT OVERWRITE TABLE hive_hbasetable_k SELECT * FROM pokes WHERE >foo=98; >Total MapReduce jobs = 1 >Launching Job 1 out of 1 >Number of reduce tasks is set to 0 since there's no reduce operator >Starting Job = job_201207250246_0005, Tracking URL = >http://0.0.0.0:50030/jobdetails.jsp?jobid=job_201207250246_0005 >Kill Command = /usr/lib/hadoop-0.20/bin/hadoop job >-Dmapred.job.tracker=0.0.0.0:8021 -kill job_201207250246_0005 >2012-07-25 04:26:00,198 Stage-0 map = 0%, reduce = 0% >2012-07-25 04:27:00,767 Stage-0 map = 0%, reduce = 0% >2012-07-25 04:27:08,844 Stage-0 map = 100%, reduce = 100% >Ended Job = job_201207250246_0005 with errors >FAILED: Execution Error, return code 2 from >org.apache.hadoop.hive.ql.exec.MapRedTask >hive> >
