Hi, I am trying to run a simple script with hcatalog and am running into issues.
Here is the pig script test.pig deepika_data = load 'hivetest' using org.apache.hcatalog.pig.HCatLoader(); dump deepika_data; I run it using this on command line - pig -Dpig.additional.jars=$PIG_CLASSPATH test.pig I get the following exception when I run the command: HBASE_HOME isn't set: $HBASE_HOME defaulting to /usr/lib/hbase 2013-05-22 13:57:33,528 [main] INFO org.apache.pig.Main - Apache Pig version 0.10.0-cdh3u4a (rexported) compiled Sep 04 2012, 14:03:46 2013-05-22 13:57:33,529 [main] INFO org.apache.pig.Main - Logging error messages to: /x/home/dkhera/scripts/pig/pig_1369256253526.log 2013-05-22 13:57:33,866 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://my-namenode:8020/ 2013-05-22 13:57:33,956 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: my-jobtracker:8021 2013-05-22 13:57:34,874 [main] ERROR org.apache.hadoop.hive.conf.HiveConf - Found both hive.metastore.uris and javax.jdo.option.ConnectionURL Recommended to have exactly one of those config keyin configuration 2013-05-22 13:57:34,979 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://my-hive-metastore:3116 2013-05-22 13:57:34,987 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2998: Unhandled internal error. java.lang.IncompatibleClassChangeError: class com.facebook.fb303.FacebookService$Client has interface org.apache.thrift.TServiceClient as super class PIG_CLASSPATH looks like: HCAT_HOME/share/hcatalog/hcatalog-pig-adapter-0.5.0-incubating.jar:\ $HCAT_HOME/share/hcatalog/storage-handlers/hbase/lib/hbase-storage-handler-0.1.0.jar:\ $HIVE_HOME/lib/hive-metastore-0.9.0-cdh3u4b-SNAPSHOT.jar:\ $HIVE_HOME/lib/hive-exec-0.9.0-cdh3u4b-SNAPSHOT.jar:\ $HIVE_HOME/lib/jdo2-api-2.3-ec.jar:\ $HIVE_HOME/conf:$HADOOP_HOME/conf:\ $HIVE_HOME/lib/libfb303-0.7.0.jar:\ $HIVE_HOME/lib/slf4j-api-1.6.1.jar export PIG_OPTS=-Dhive.metastore.uris=thrift://my-hive-metastore:3116 This is as described here http://hive.apache.org/docs/hcat_r0.5.0/loadstore.html When I use the debug flag on pig, I get HADOOP_CLASSPATH: /usr/lib/hbase/conf:/usr/lib/pig/bin/../conf:/usr/java/jdk1.6.0_27/lib/tools.jar:/usr/lib/hbase/hbase.jar:/usr/lib/hbase/lib/*:/usr/lib/hadoop/conf:/usr/lib/hbase/conf:/usr/lib/pig/bin/../conf:/x/home/dkhera/hcat-client/hcatalog-incubating/share/hcatalog/hcatalog-core-0.5.0-incubating.jar:/x/home/dkhera/hcat-client/hcatalog-incubating/share/hcatalog/hcatalog-pig-adapter-0.5.0-incubating.jar:/x/home/dkhera/hcat-client/hcatalog-incubating/share/hcatalog/storage-handlers/hbase/lib/hbase-storage-handler-0.1.0.jar:/usr/lib/hive/lib/hive-metastore-0.9.0-cdh3u4b-SNAPSHOT.jar:/usr/lib/hive/lib/hive-exec-0.9.0-cdh3u4b-SNAPSHOT.jar:/usr/lib/hive/lib/jdo2-api-2.3-ec.jar:/usr/lib/hive/conf:/usr/lib/hadoop/conf:/usr/lib/hive/lib/libfb303-0.7.0.jar:/usr/lib/hive/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop/conf:/usr/lib/pig/bin/../lib/automaton.jar:/usr/lib/pig/bin/../lib/avro-1.5.4.jar:/usr/lib/pig/bin/../lib/jackson-core-asl-1.7.3.jar:/usr/lib/pig/bin/../lib/jackson-mapper-asl-1.7.3.jar:/usr/lib/pig/bin/../lib/json-simple-1.1.jar:/usr/lib/pig/bin/../lib/jython-2.5.0.jar:/usr/lib/pig/bin/../lib/snappy-java-1.0.3.2.jar:/usr/lib/pig/bin/../pig-0.10.0-cdh3u4a-withouthadoop.jar: HADOOP_OPTS: -Xmx1000m -Dhive.metastore.uris=thrift://horton-mysql.vip.lvs.paypal.com:3116 -Djava.library.path=/usr/lib/hadoop/lib/native/Linux-amd64-64 -Djava.io.tmpdir=/x/cdh/pig/tmp -Dpig.log.dir=/usr/lib/pig/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/usr/lib/pig/bin/.. /usr/bin/hadoop jar /usr/lib/pig/bin/../pig-0.10.0-cdh3u4a-withouthadoop.jar test.pig Note that libthrift-0.7.0.jar is in hbase lib (already in class path), so I did not explicitly add it in pig class path. Does anyone know what is causing this. It certainly seems like a version conflict of thrift jars, but I am unable to figure it out yet. Thanks, Deepika
