Hi all. I have some problem with hive-on-tez. email thread below is forwarding originally wrote to tez users.
Could someone please check this problem out? Any advice will be appreciated. thanks. ---------- Forwarded message ---------- From: no jihun <jees...@gmail.com> Date: 2016-02-15 17:04 GMT+09:00 Subject: Re: Failing attemption at org.apache.tez.client.TezClient.waitTillReady To: u...@tez.apache.org Thanks Hitesh Shah. btw. I am sorry but It is very wired. I can't find any previous version files of hadoop version conflict on any node. I've tried minimal.tar.gz and tez.use.cluster.hadoop-libs, but result was same. With my set up the examples of tez wiki seems work well. *$ ~/apps/hadoop/bin/hadoop jar tez-dist/target/tez-0.8.2/tez-tests-0.8.2.jar testorderedwordcount -DUSE_TEZ_SESSION=true /tmp/wc1 /tmp/wc1.r /tmp/wc2 /tmp/wc2.r* *SLF4J: Class path contains multiple SLF4J bindings.* *SLF4J: Found binding in [jar:file:/home1/irteam/apps/tez-0.8.2/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]* *SLF4J: Found binding in [jar:file:/home1/irteam/apps/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]* *.....* * Shuffle Errors* * BAD_ID=0* * CONNECTION=0* * IO_ERROR=0* * WRONG_LENGTH=0* * WRONG_MAP=0* * WRONG_REDUCE=0* *16/02/15 16:27:51 INFO examples.TestOrderedWordCount: DAG 2 completed. FinalState=SUCCEEDED* *16/02/15 16:27:51 INFO examples.TestOrderedWordCount: Shutting down session* *16/02/15 16:27:51 INFO client.TezClient: Shutting down Tez Session, sessionName=OrderedWordCountSession, applicationId=application_1452243782005_0323* The method with problem seems added from hadoop 2.4. *org.apache.hadoop.conf.Configuration.addDeprecations([Lorg/apache/hadoop/conf/Configuration$DeprecationDelta;)* hadoop-common jars.. *$ find ./ -name hadoop-common*jar* *./apps/tez-0.8.2/lib/hadoop-common-2.6.0.jar* *./apps/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0.jar* *./apps/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0-tests.jar* *./apps/hadoop-2.6.0/share/hadoop/common/sources/hadoop-common-2.6.0-sources.jar* *./apps/hadoop-2.6.0/share/hadoop/common/sources/hadoop-common-2.6.0-test-sources.jar* *./apps/hadoop-2.6.0/share/hadoop/httpfs/tomcat/webapps/webhdfs/WEB-INF/lib/hadoop-common-2.6.0.jar* *./apps/hadoop-2.6.0/share/hadoop/kms/tomcat/webapps/kms/WEB-INF/lib/hadoop-common-2.6.0.jar* When I print jinfo of hive cli process there is no other version of hadoop jars... *$ jinfo 52974* *Attaching to process ID 52974, please wait...* *Debugger attached successfully.* *Server compiler detected.* *JVM version is 25.60-b23* *Java System Properties* *java.runtime.name <http://java.runtime.name/> = Java(TM) SE Runtime Environment* *java.vm.version = 25.60-b23* *sun.boot.library.path = /home1/apps/jdk1.8.0_60/jre/lib/amd64* *hadoop.root.logger = INFO,console* *java.vendor.url = http* *//java.oracle.com/ <http://java.oracle.com/>* *java.vm.vendor = Oracle Corporation* *path.separator = * *file.encoding.pkg = sun.io <http://sun.io/>* *java.vm.name <http://java.vm.name/> = Java HotSpot(TM) 64-Bit Server VM* *sun.os.patch.level = unknown* *sun.java.launcher = SUN_STANDARD* *user.country = KR* *user.dir = /home1/apps/apache-hive-1.2.1-bin/bin* *java.vm.specification.name <http://java.vm.specification.name/> = Java Virtual Machine Specification* *java.runtime.version = 1.8.0_60-b27* *java.awt.graphicsenv = sun.awt.X11GraphicsEnvironment* *os.arch = amd64* *java.endorsed.dirs = /home1/apps/jdk1.8.0_60/jre/lib/endorsed* *line.separator = * *java.io.tmpdir = /tmp* *hadoop.log.file = hadoop.log* *java.vm.specification.vendor = Oracle Corporation* *os.name <http://os.name/> = Linux* *hadoop.id.str = irteam* *sun.jnu.encoding = UTF-8* *java.library.path = /home1/apps/hadoop/lib/native* *hadoop.home.dir = /home1/apps/hadoop* *java.specification.name <http://java.specification.name/> = Java Platform API Specification* *java.class.version = 52.0* *java.net.preferIPv4Stack = true* *sun.management.compiler = HotSpot 64-Bit Tiered Compilers* *os.version = 2.6.32-504.3.3.el6.x86_64* *user.home = /home1* *user.timezone = ROK* *java.awt.printerjob = sun.print.PSPrinterJob* *file.encoding = UTF-8* *java.specification.version = 1.8* *user.name <http://user.name/> = irteam* *java.class.path = /home1/apps/tez-0.8.2/conf* */home1/apps/tez-0.8.2/tez-history-parser-0.8.2.jar* */home1/apps/tez-0.8.2/tez-yarn-timeline-history-with-acls-0.8.2.jar* */home1/apps/tez-0.8.2/tez-examples-0.8.2.jar* */home1/apps/tez-0.8.2/tez-javadoc-tools-0.8.2.jar* */home1/apps/tez-0.8.2/tez-runtime-internals-0.8.2.jar* */home1/apps/tez-0.8.2/hadoop-shim-2.6-0.8.2.jar* */home1/apps/tez-0.8.2/tez-runtime-library-0.8.2.jar* */home1/apps/tez-0.8.2/tez-job-analyzer-0.8.2.jar* */home1/apps/tez-0.8.2/tez-yarn-timeline-history-0.8.2.jar* */home1/apps/tez-0.8.2/tez-api-0.8.2.jar* */home1/apps/tez-0.8.2/tez-mapreduce-0.8.2.jar* */home1/apps/tez-0.8.2/hadoop-shim-0.8.2.jar* */home1/apps/tez-0.8.2/tez-common-0.8.2.jar* */home1/apps/tez-0.8.2/tez-tests-0.8.2.jar* */home1/apps/tez-0.8.2/tez-ext-service-tests-0.8.2.jar* */home1/apps/tez-0.8.2/tez-dag-0.8.2.jar* */home1/apps/tez-0.8.2/lib/xmlenc-0.52.jar* */home1/apps/tez-0.8.2/lib/commons-collections4-4.1.jar* */home1/apps/tez-0.8.2/lib/httpclient-4.2.5.jar* */home1/apps/tez-0.8.2/lib/jetty-6.1.26.jar* */home1/apps/tez-0.8.2/lib/commons-codec-1.4.jar* */home1/apps/tez-0.8.2/lib/curator-recipes-2.6.0.jar* */home1/apps/tez-0.8.2/lib/RoaringBitmap-0.4.9.jar* */home1/apps/tez-0.8.2/lib/jaxb-api-2.2.2.jar* */home1/apps/tez-0.8.2/lib/xz-1.0.jar* */home1/apps/tez-0.8.2/lib/hadoop-mapreduce-client-common-2.6.0.jar* */home1/apps/tez-0.8.2/lib/activation-1.1.jar* */home1/apps/tez-0.8.2/lib/hadoop-common-2.6.0.jar* */home1/apps/tez-0.8.2/lib/javax.inject-1.jar* */home1/apps/tez-0.8.2/lib/commons-math3-3.1.1.jar* */home1/apps/tez-0.8.2/lib/jersey-json-1.9.jar* */home1/apps/tez-0.8.2/lib/httpcore-4.2.4.jar* */home1/apps/tez-0.8.2/lib/jersey-core-1.9.jar* */home1/apps/tez-0.8.2/lib/commons-io-2.4.jar* */home1/apps/tez-0.8.2/lib/servlet-api-2.5.jar* */home1/apps/tez-0.8.2/lib/hadoop-auth-2.6.0.jar* */home1/apps/tez-0.8.2/lib/hadoop-hdfs-2.6.0.jar* */home1/apps/tez-0.8.2/lib/metrics-core-3.1.0.jar* */home1/apps/tez-0.8.2/lib/protobuf-java-2.5.0.jar* */home1/apps/tez-0.8.2/lib/hadoop-yarn-server-web-proxy-2.6.0.jar* */home1/apps/tez-0.8.2/lib/guava-11.0.2.jar* */home1/apps/tez-0.8.2/lib/jackson-core-asl-1.9.13.jar* */home1/apps/tez-0.8.2/lib/apacheds-i18n-2.0.0-M15.jar* */home1/apps/tez-0.8.2/lib/api-util-1.0.0-M20.jar* */home1/apps/tez-0.8.2/lib/commons-compress-1.4.1.jar* */home1/apps/tez-0.8.2/lib/jetty-util-6.1.26.jar* */home1/apps/tez-0.8.2/lib/curator-client-2.6.0.jar* */home1/apps/tez-0.8.2/lib/snappy-java-1.0.4.1.jar* */home1/apps/tez-0.8.2/lib/slf4j-api-1.7.10.jar* */home1/apps/tez-0.8.2/lib/xercesImpl-2.9.1.jar* */home1/apps/tez-0.8.2/lib/hadoop-yarn-common-2.6.0.jar* */home1/apps/tez-0.8.2/lib/jersey-client-1.9.jar* */home1/apps/tez-0.8.2/lib/apacheds-kerberos-codec-2.0.0-M15.jar* */home1/apps/tez-0.8.2/lib/jsr305-3.0.0.jar* */home1/apps/tez-0.8.2/lib/jettison-1.3.4.jar* */home1/apps/tez-0.8.2/lib/log4j-1.2.17.jar* */home1/apps/tez-0.8.2/lib/stax-api-1.0.1.jar* */home1/apps/tez-0.8.2/lib/hadoop-yarn-server-common-2.6.0.jar* */home1/apps/tez-0.8.2/lib/zookeeper-3.4.6.jar* */home1/apps/tez-0.8.2/lib/commons-configuration-1.6.jar* */home1/apps/tez-0.8.2/lib/stax-api-1.0-2.jar* */home1/apps/tez-0.8.2/lib/guice-3.0.jar* */home1/apps/tez-0.8.2/lib/hadoop-yarn-api-2.6.0.jar* */home1/apps/tez-0.8.2/lib/leveldbjni-all-1.8.jar* */home1/apps/tez-0.8.2/lib/gson-2.2.4.jar* */home1/apps/tez-0.8.2/lib/htrace-core-3.0.4.jar* */home1/apps/tez-0.8.2/lib/jackson-mapper-asl-1.9.13.jar* */home1/apps/tez-0.8.2/lib/jackson-jaxrs-1.9.13.jar* */home1/apps/tez-0.8.2/lib/avro-1.7.4.jar* */home1/apps/tez-0.8.2/lib/servlet-api-2.5-20081211.jar* */home1/apps/tez-0.8.2/lib/jersey-guice-1.9.jar* */home1/apps/tez-0.8.2/lib/aopalliance-1.0.jar* */home1/apps/tez-0.8.2/lib/hadoop-annotations-2.6.0.jar* */home1/apps/tez-0.8.2/lib/xml-apis-1.3.04.jar* */home1/apps/tez-0.8.2/lib/netty-3.9.2.Final.jar* */home1/apps/tez-0.8.2/lib/curator-framework-2.6.0.jar* */home1/apps/tez-0.8.2/lib/async-http-client-1.8.16.jar* */home1/apps/tez-0.8.2/lib/commons-digester-1.8.jar* */home1/apps/tez-0.8.2/lib/paranamer-2.3.jar* */home1/apps/tez-0.8.2/lib/hadoop-mapreduce-client-core-2.6.0.jar* */home1/apps/tez-0.8.2/lib/jaxb-impl-2.2.3-1.jar* */home1/apps/tez-0.8.2/lib/hadoop-yarn-client-2.6.0.jar* */home1/apps/tez-0.8.2/lib/jackson-xc-1.9.13.jar* */home1/apps/tez-0.8.2/lib/api-asn1-api-1.0.0-M20.jar* */home1/apps/tez-0.8.2/lib/slf4j-log4j12-1.7.10.jar* */home1/apps/tez-0.8.2/lib/commons-collections-3.2.2.jar* */home1/apps/tez-0.8.2/lib/commons-beanutils-1.7.0.jar* */home1/apps/tez-0.8.2/lib/commons-cli-1.2.jar* */home1/apps/tez-0.8.2/lib/commons-lang-2.6.jar* */home1/apps/tez-0.8.2/lib/commons-beanutils-core-1.8.0.jar* */home1/apps/tez-0.8.2/lib/guice-servlet-3.0.jar* */home1/apps/tez-0.8.2/lib/commons-httpclient-3.1.jar* */home1/apps/tez-0.8.2/lib/commons-logging-1.1.3.jar* */home1/apps/tez-0.8.2/lib/commons-net-3.1.jar* */home1/apps/jdk/lib/tools.jar* */home1/eco/hive/conf* */home1/apps/hive/lib/ST4-4.0.4.jar* */home1/apps/hive/lib/accumulo-core-1.6.0.jar* */home1/apps/hive/lib/accumulo-fate-1.6.0.jar* */home1/apps/hive/lib/accumulo-start-1.6.0.jar* */home1/apps/hive/lib/accumulo-trace-1.6.0.jar* */home1/apps/hive/lib/activation-1.1.jar* */home1/apps/hive/lib/ant-1.9.1.jar* */home1/apps/hive/lib/ant-launcher-1.9.1.jar* */home1/apps/hive/lib/antlr-2.7.7.jar* */home1/apps/hive/lib/antlr-runtime-3.4.jar* */home1/apps/hive/lib/apache-log4j-extras-1.2.17.jar* */home1/apps/hive/lib/asm-commons-3.1.jar* */home1/apps/hive/lib/asm-tree-3.1.jar* */home1/apps/hive/lib/avro-1.7.5.jar* */home1/apps/hive/lib/bonecp-0.8.0.RELEASE.jar* */home1/apps/hive/lib/calcite-avatica-1.2.0-incubating.jar* */home1/apps/hive/lib/calcite-core-1.2.0-incubating.jar* */home1/apps/hive/lib/calcite-linq4j-1.2.0-incubating.jar* */home1/apps/hive/lib/commons-beanutils-1.7.0.jar* */home1/apps/hive/lib/commons-beanutils-core-1.8.0.jar* */home1/apps/hive/lib/commons-cli-1.2.jar* */home1/apps/hive/lib/commons-codec-1.4.jar* */home1/apps/hive/lib/commons-collections-3.2.1.jar* */home1/apps/hive/lib/commons-compiler-2.7.6.jar* */home1/apps/hive/lib/commons-compress-1.4.1.jar* */home1/apps/hive/lib/commons-configuration-1.6.jar* */home1/apps/hive/lib/commons-dbcp-1.4.jar* */home1/apps/hive/lib/commons-digester-1.8.jar* */home1/apps/hive/lib/commons-httpclient-3.0.1.jar* */home1/apps/hive/lib/commons-io-2.4.jar* */home1/apps/hive/lib/commons-lang-2.6.jar* */home1/apps/hive/lib/commons-logging-1.1.3.jar* */home1/apps/hive/lib/commons-math-2.1.jar* */home1/apps/hive/lib/commons-pool-1.5.4.jar* */home1/apps/hive/lib/commons-vfs2-2.0.jar* */home1/apps/hive/lib/curator-client-2.6.0.jar* */home1/apps/hive/lib/curator-framework-2.6.0.jar* */home1/apps/hive/lib/curator-recipes-2.6.0.jar* */home1/apps/hive/lib/datanucleus-api-jdo-3.2.6.jar* */home1/apps/hive/lib/datanucleus-core-3.2.10.jar* */home1/apps/hive/lib/datanucleus-rdbms-3.2.9.jar* */home1/apps/hive/lib/derby-10.10.2.0.jar* */home1/apps/hive/lib/eigenbase-properties-1.1.5.jar* */home1/apps/hive/lib/geronimo-annotation_1.0_spec-1.1.1.jar* */home1/apps/hive/lib/geronimo-jaspic_1.0_spec-1.0.jar* */home1/apps/hive/lib/geronimo-jta_1.1_spec-1.1.1.jar* */home1/apps/hive/lib/groovy-all-2.1.6.jar* */home1/apps/hive/lib/guava-14.0.1.jar* */home1/apps/hive/lib/hamcrest-core-1.1.jar* */home1/apps/hive/lib/hive-accumulo-handler-1.2.1.jar* */home1/apps/hive/lib/hive-ant-1.2.1.jar* */home1/apps/hive/lib/hive-beeline-1.2.1.jar* */home1/apps/hive/lib/hive-cli-1.2.1.jar* */home1/apps/hive/lib/hive-common-1.2.1.jar* */home1/apps/hive/lib/hive-contrib-1.2.1.jar* */home1/apps/hive/lib/hive-exec-1.2.1.jar* */home1/apps/hive/lib/hive-hbase-handler-1.2.1.jar* */home1/apps/hive/lib/hive-hwi-1.2.1.jar* */home1/apps/hive/lib/hive-jdbc-1.2.1-standalone.jar* */home1/apps/hive/lib/hive-jdbc-1.2.1.jar* */home1/apps/hive/lib/hive-metastore-1.2.1.jar* */home1/apps/hive/lib/hive-serde-1.2.1.jar* */home1/apps/hive/lib/hive-service-1.2.1.jar* */home1/apps/hive/lib/hive-shims-0.20S-1.2.1.jar* */home1/apps/hive/lib/hive-shims-0.23-1.2.1.jar* */home1/apps/hive/lib/hive-shims-1.2.1.jar* */home1/apps/hive/lib/hive-shims-common-1.2.1.jar* */home1/apps/hive/lib/hive-shims-scheduler-1.2.1.jar* */home1/apps/hive/lib/hive-testutils-1.2.1.jar* */home1/apps/hive/lib/httpclient-4.4.jar* */home1/apps/hive/lib/httpcore-4.4.jar* */home1/apps/hive/lib/ivy-2.4.0.jar* */home1/apps/hive/lib/janino-2.7.6.jar* */home1/apps/hive/lib/jcommander-1.32.jar* */home1/apps/hive/lib/jdo-api-3.0.1.jar* */home1/apps/hive/lib/jetty-all-7.6.0.v20120127.jar* */home1/apps/hive/lib/jetty-all-server-7.6.0.v20120127.jar* */home1/apps/hive/lib/jline-2.12.jar* */home1/apps/hive/lib/joda-time-2.5.jar* */home1/apps/hive/lib/jpam-1.1.jar* */home1/apps/hive/lib/json-20090211.jar* */home1/apps/hive/lib/jsr305-3.0.0.jar* */home1/apps/hive/lib/jta-1.1.jar* */home1/apps/hive/lib/junit-4.11.jar* */home1/apps/hive/lib/libfb303-0.9.2.jar* */home1/apps/hive/lib/libthrift-0.9.2.jar* */home1/apps/hive/lib/log4j-1.2.16.jar* */home1/apps/hive/lib/mail-1.4.1.jar* */home1/apps/hive/lib/maven-scm-api-1.4.jar* */home1/apps/hive/lib/maven-scm-provider-svn-commons-1.4.jar* */home1/apps/hive/lib/maven-scm-provider-svnexe-1.4.jar* */home1/apps/hive/lib/netty-3.7.0.Final.jar* */home1/apps/hive/lib/opencsv-2.3.jar* */home1/apps/hive/lib/oro-2.0.8.jar* */home1/apps/hive/lib/paranamer-2.3.jar* */home1/apps/hive/lib/parquet-hadoop-bundle-1.6.0.jar* */home1/apps/hive/lib/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar* */home1/apps/hive/lib/plexus-utils-1.5.6.jar* */home1/apps/hive/lib/regexp-1.3.jar* */home1/apps/hive/lib/servlet-api-2.5.jar* */home1/apps/hive/lib/snappy-java-1.0.5.jar* */home1/apps/hive/lib/spark-assembly-1.4.1-hadoop2.6.0.jar* */home1/apps/hive/lib/stax-api-1.0.1.jar* */home1/apps/hive/lib/stringtemplate-3.2.1.jar* */home1/apps/hive/lib/super-csv-2.2.0.jar* */home1/apps/hive/lib/tempus-fugit-1.1.jar* */home1/apps/hive/lib/velocity-1.5.jar* */home1/apps/hive/lib/xz-1.0.jar* */home1/apps/hive/lib/zookeeper-3.4.6.jar* */home1/apps/spark-1.4.1-bin-hadoop2.6/lib/spark-assembly-1.4.1-hadoop2.6.0.jar* */home1/eco/hive/aux_lib/hive-json-serde-0.3.jar* */home1/eco/hive/aux_lib/mysql-connector-java-5.1.34.jar* */home1/eco/hive/aux_lib/rcongiu-jsonserde-1.3.jar* */home1/apps/hadoop/contrib/capacity-scheduler/*.jar* */home1/eco/hadoop/conf* */home1/apps/hadoop/share/hadoop/common/lib/jasper-compiler-5.5.23.jar* */home1/apps/hadoop/share/hadoop/common/lib/xmlenc-0.52.jar* */home1/apps/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar* */home1/apps/hadoop/share/hadoop/common/lib/httpclient-4.2.5.jar* */home1/apps/hadoop/share/hadoop/common/lib/jetty-6.1.26.jar* */home1/apps/hadoop/share/hadoop/common/lib/commons-codec-1.4.jar* */home1/apps/hadoop/share/hadoop/common/lib/curator-recipes-2.6.0.jar* */home1/apps/hadoop/share/hadoop/common/lib/jaxb-api-2.2.2.jar* */home1/apps/hadoop/share/hadoop/common/lib/asm-3.2.jar* */home1/apps/hadoop/share/hadoop/common/lib/jersey-server-1.9.jar* */home1/apps/hadoop/share/hadoop/common/lib/xz-1.0.jar* */home1/apps/hadoop/share/hadoop/common/lib/activation-1.1.jar* */home1/apps/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar* */home1/apps/hadoop/share/hadoop/common/lib/jersey-json-1.9.jar* */home1/apps/hadoop/share/hadoop/common/lib/jersey-core-1.9.jar* */home1/apps/hadoop/share/hadoop/common/lib/commons-io-2.4.jar* */home1/apps/hadoop/share/hadoop/common/lib/servlet-api-2.5.jar* */home1/apps/hadoop/share/hadoop/common/lib/hadoop-auth-2.6.0.jar* */home1/apps/hadoop/share/hadoop/common/lib/protobuf-java-2.5.0.jar* */home1/apps/hadoop/share/hadoop/common/lib/junit-4.11.jar* */home1/apps/hadoop/share/hadoop/common/lib/guava-11.0.2.jar* */home1/apps/hadoop/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar* */home1/apps/hadoop/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar* */home1/apps/hadoop/share/hadoop/common/lib/api-util-1.0.0-M20.jar* */home1/apps/hadoop/share/hadoop/common/lib/commons-compress-1.4.1.jar* */home1/apps/hadoop/share/hadoop/common/lib/jetty-util-6.1.26.jar* */home1/apps/hadoop/share/hadoop/common/lib/curator-client-2.6.0.jar* */home1/apps/hadoop/share/hadoop/common/lib/snappy-java-1.0.4.1.jar* */home1/apps/hadoop/share/hadoop/common/lib/netty-3.6.2.Final.jar* */home1/apps/hadoop/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar* */home1/apps/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar* */home1/apps/hadoop/share/hadoop/common/lib/mockito-all-1.8.5.jar* */home1/apps/hadoop/share/hadoop/common/lib/jasper-runtime-5.5.23.jar* */home1/apps/hadoop/share/hadoop/common/lib/zookeeper-3.4.6.jar* */home1/apps/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar* */home1/apps/hadoop/share/hadoop/common/lib/jets3t-0.9.0.jar* */home1/apps/hadoop/share/hadoop/common/lib/stax-api-1.0-2.jar* */home1/apps/hadoop/share/hadoop/common/lib/gson-2.2.4.jar* */home1/apps/hadoop/share/hadoop/common/lib/hamcrest-core-1.3.jar* */home1/apps/hadoop/share/hadoop/common/lib/commons-collections-3.2.1.jar* */home1/apps/hadoop/share/hadoop/common/lib/htrace-core-3.0.4.jar* */home1/apps/hadoop/share/hadoop/common/lib/slf4j-api-1.7.5.jar* */home1/apps/hadoop/share/hadoop/common/lib/java-xmlbuilder-0.4.jar* */home1/apps/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar* */home1/apps/hadoop/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar* */home1/apps/hadoop/share/hadoop/common/lib/jsch-0.1.42.jar* */home1/apps/hadoop/share/hadoop/common/lib/avro-1.7.4.jar* */home1/apps/hadoop/share/hadoop/common/lib/jettison-1.1.jar* */home1/apps/hadoop/share/hadoop/common/lib/hadoop-annotations-2.6.0.jar* */home1/apps/hadoop/share/hadoop/common/lib/jsr305-1.3.9.jar* */home1/apps/hadoop/share/hadoop/common/lib/curator-framework-2.6.0.jar* */home1/apps/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar* */home1/apps/hadoop/share/hadoop/common/lib/httpcore-4.2.5.jar* */home1/apps/hadoop/share/hadoop/common/lib/commons-digester-1.8.jar* */home1/apps/hadoop/share/hadoop/common/lib/paranamer-2.3.jar* */home1/apps/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar* */home1/apps/hadoop/share/hadoop/common/lib/commons-el-1.0.jar* */home1/apps/hadoop/share/hadoop/common/lib/jackson-xc-1.9.13.jar* */home1/apps/hadoop/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar* */home1/apps/hadoop/share/hadoop/common/lib/commons-beanutils-1.7.0.jar* */home1/apps/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar* */home1/apps/hadoop/share/hadoop/common/lib/commons-lang-2.6.jar* */home1/apps/hadoop/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar* */home1/apps/hadoop/share/hadoop/common/lib/commons-httpclient-3.1.jar* */home1/apps/hadoop/share/hadoop/common/lib/commons-logging-1.1.3.jar* */home1/apps/hadoop/share/hadoop/common/lib/commons-net-3.1.jar* */home1/apps/hadoop/share/hadoop/common/hadoop-common-2.6.0.jar* */home1/apps/hadoop/share/hadoop/common/hadoop-common-2.6.0-tests.jar* */home1/apps/hadoop/share/hadoop/common/hadoop-nfs-2.6.0.jar* */home1/apps/hadoop/share/hadoop/hdfs* */home1/apps/hadoop/share/hadoop/hdfs/lib/xmlenc-0.52.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/jsp-api-2.1.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/jetty-6.1.26.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/commons-codec-1.4.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/asm-3.2.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/jersey-server-1.9.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/jersey-core-1.9.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/commons-io-2.4.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/servlet-api-2.5.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/guava-11.0.2.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/log4j-1.2.17.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/htrace-core-3.0.4.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/jsr305-1.3.9.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/commons-el-1.0.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/commons-lang-2.6.jar* */home1/apps/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar* */home1/apps/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.6.0.jar* */home1/apps/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.6.0-tests.jar* */home1/apps/hadoop/share/hadoop/hdfs/hadoop-hdfs-nfs-2.6.0.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/jetty-6.1.26.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/commons-codec-1.4.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/asm-3.2.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/jersey-server-1.9.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/xz-1.0.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/activation-1.1.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/javax.inject-1.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/jersey-json-1.9.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/jersey-core-1.9.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/commons-io-2.4.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/servlet-api-2.5.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/jline-0.9.94.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/guava-11.0.2.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/commons-compress-1.4.1.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/jetty-util-6.1.26.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/netty-3.6.2.Final.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/jersey-client-1.9.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/log4j-1.2.17.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/zookeeper-3.4.6.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/stax-api-1.0-2.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/guice-3.0.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/commons-collections-3.2.1.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/jersey-guice-1.9.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/jettison-1.1.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/aopalliance-1.0.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/jsr305-1.3.9.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/commons-cli-1.2.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/commons-lang-2.6.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/guice-servlet-3.0.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/commons-httpclient-3.1.jar* */home1/apps/hadoop/share/hadoop/yarn/lib/commons-logging-1.1.3.jar* */home1/apps/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.6.0.jar* */home1/apps/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.6.0.jar* */home1/apps/hadoop/share/hadoop/yarn/hadoop-yarn-registry-2.6.0.jar* */home1/apps/hadoop/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.6.0.jar* */home1/apps/hadoop/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar* */home1/apps/hadoop/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.6.0.jar* */home1/apps/hadoop/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.6.0.jar* */home1/apps/hadoop/share/hadoop/yarn/hadoop-yarn-common-2.6.0.jar* */home1/apps/hadoop/share/hadoop/yarn/hadoop-yarn-server-common-2.6.0.jar* */home1/apps/hadoop/share/hadoop/yarn/hadoop-yarn-api-2.6.0.jar* */home1/apps/hadoop/share/hadoop/yarn/hadoop-yarn-client-2.6.0.jar* */home1/apps/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-2.6.0.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/asm-3.2.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/jersey-server-1.9.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/xz-1.0.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/javax.inject-1.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/jersey-core-1.9.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/commons-io-2.4.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/junit-4.11.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/log4j-1.2.17.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/guice-3.0.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/avro-1.7.4.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/aopalliance-1.0.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/hadoop-annotations-2.6.0.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/paranamer-2.3.jar* */home1/apps/hadoop/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar* */home1/apps/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.6.0.jar* */home1/apps/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.6.0.jar* */home1/apps/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.6.0.jar* */home1/apps/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.6.0.jar* */home1/apps/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar* */home1/apps/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.6.0.jar* */home1/apps/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0.jar* */home1/apps/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.6.0.jar* */home1/apps/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-tests.jar* *java.vm.specification.version = 1.8* *sun.arch.data.model = 64* *sun.java.command = org.apache.hadoop.util.RunJar /home1/apps/hive/lib/hive-service-1.2.1.jar org.apache.hadoop.hive.metastore.HiveMetaStore* *java.home = /home1/apps/jdk1.8.0_60/jre* *user.language = ko* *java.specification.vendor = Oracle Corporation* *hadoop.security.logger = INFO,NullAppender* *awt.toolkit = sun.awt.X11.XToolkit* *java.vm.info <http://java.vm.info/> = mixed mode* *hadoop.log.dir = /home1/logs/hadoop/* *java.version = 1.8.0_60* *java.ext.dirs = /home1/apps/jdk1.8.0_60/jre/lib/ext* */usr/java/packages/lib/ext* *sun.boot.class.path = /home1/apps/jdk1.8.0_60/jre/lib/resources.jar* */home1/apps/jdk1.8.0_60/jre/lib/rt.jar* */home1/apps/jdk1.8.0_60/jre/lib/sunrsasign.jar* */home1/apps/jdk1.8.0_60/jre/lib/jsse.jar* */home1/apps/jdk1.8.0_60/jre/lib/jce.jar* */home1/apps/jdk1.8.0_60/jre/lib/charsets.jar* */home1/apps/jdk1.8.0_60/jre/lib/jfr.jar* */home1/apps/jdk1.8.0_60/jre/classes* *java.vendor = Oracle Corporation* *file.separator = /* *java.vendor.url.bug = http* *//bugreport.sun.com/bugreport/ <http://bugreport.sun.com/bugreport/>* *sun.io.unicode.encoding = UnicodeLittle* *sun.cpu.endian = little* *hadoop.policy.file = hadoop-policy.xml* *sun.cpu.isalist = * *VM Flags* *Non-default VM flags* * -XX* *CICompilerCount=12 -XX* *InitialHeapSize=262144000 -XX* *MaxHeapSize=536870912 -XX* *MaxNewSize=178782208 -XX* *MinHeapDeltaBytes=524288 -XX* *NewSize=87031808 -XX* *OldSize=175112192 -XX* *+UseCompressedClassPointers -XX* *+UseCompressedOops -XX* *+UseFastUnorderedTimeStamps -XX* *+UseParallelGC * *Command line* * -Xmx256m -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/home1/logs/hadoop/ -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/home1/apps/hadoop -Dhadoop.id.str=irteam -Dhadoop.root.logger=INFO,console -Djava.library.path=/home1/apps/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xmx512m -Dhadoop.security.logger=INFO,NullAppender* Anyway, I think It is engaged with hive. because tez-test and tez-example which not related with hive works well. any help..? 2016-02-13 4:31 GMT+09:00 Hitesh Shah <hit...@apache.org>: > I think there is a mix of classes of hadoop-common getting on to the > classpath. This might possibly be due to hive-exec.jar. > > One option for you to try is put the tez-minimal tarball on HDFS instead > of the full tarball and set “tez.use.cluster.hadoop-libs” to true in your > tez-site.xml. This will use hadoop jars from the cluster and hopefully > solve the mixed version problem. > > thanks > — Hitesh > > > On Feb 12, 2016, at 1:01 AM, no jihun <jees...@gmail.com> wrote: > > > * I did nothing with hive-exec.jar, hive.jar.directory.. > > > > * hive-default.xml > > > > $ cat eco/hive/conf/hive-default.xml > > > > <property> > > <name>hive.jar.directory</name> > > <value/> > > <description> > > This is the location hive in tez mode will look for to find a site > wide > > installed hive instance. > > </description> > > </property> > > > > > > * tez/lib have hadoop-common-2.6.0.jar > > * tez tarball on hdfs also exactly same jar list > > $ pwd > > /home1/apps/tez-0.8.2/lib > > $ ll > > -rw-rw-r-- 1 124846 2016-02-11 13:37 RoaringBitmap-0.4.9.jar > > -rw-rw-r-- 1 62983 2016-02-11 13:37 activation-1.1.jar > > -rw-rw-r-- 1 4467 2016-02-11 13:37 aopalliance-1.0.jar > > -rw-rw-r-- 1 44925 2016-02-11 13:37 apacheds-i18n-2.0.0-M15.jar > > -rw-rw-r-- 1 691479 2016-02-11 13:37 > apacheds-kerberos-codec-2.0.0-M15.jar > > -rw-rw-r-- 1 16560 2016-02-11 13:37 api-asn1-api-1.0.0-M20.jar > > -rw-rw-r-- 1 79912 2016-02-11 13:37 api-util-1.0.0-M20.jar > > -rw-rw-r-- 1 535731 2016-02-11 13:37 async-http-client-1.8.16.jar > > -rw-rw-r-- 1 303139 2016-02-11 13:37 avro-1.7.4.jar > > -rw-rw-r-- 1 188671 2016-02-11 13:37 commons-beanutils-1.7.0.jar > > -rw-rw-r-- 1 206035 2016-02-11 13:37 commons-beanutils-core-1.8.0.jar > > -rw-rw-r-- 1 41123 2016-02-11 13:37 commons-cli-1.2.jar > > -rw-rw-r-- 1 58160 2016-02-11 13:37 commons-codec-1.4.jar > > -rw-rw-r-- 1 588337 2016-02-11 13:37 commons-collections-3.2.2.jar > > -rw-rw-r-- 1 751238 2016-02-11 13:37 commons-collections4-4.1.jar > > -rw-rw-r-- 1 241367 2016-02-11 13:37 commons-compress-1.4.1.jar > > -rw-rw-r-- 1 298829 2016-02-11 13:37 commons-configuration-1.6.jar > > -rw-rw-r-- 1 143602 2016-02-11 13:37 commons-digester-1.8.jar > > -rw-rw-r-- 1 305001 2016-02-11 13:37 commons-httpclient-3.1.jar > > -rw-rw-r-- 1 185140 2016-02-11 13:37 commons-io-2.4.jar > > -rw-rw-r-- 1 284220 2016-02-11 13:37 commons-lang-2.6.jar > > -rw-rw-r-- 1 62050 2016-02-11 13:37 commons-logging-1.1.3.jar > > -rw-rw-r-- 1 1599627 2016-02-11 13:37 commons-math3-3.1.1.jar > > -rw-rw-r-- 1 273370 2016-02-11 13:37 commons-net-3.1.jar > > -rw-rw-r-- 1 68866 2016-02-11 13:37 curator-client-2.6.0.jar > > -rw-rw-r-- 1 185245 2016-02-11 13:37 curator-framework-2.6.0.jar > > -rw-rw-r-- 1 248171 2016-02-11 13:37 curator-recipes-2.6.0.jar > > -rw-rw-r-- 1 190432 2016-02-11 13:37 gson-2.2.4.jar > > -rw-rw-r-- 1 1648200 2016-02-11 13:37 guava-11.0.2.jar > > -rw-rw-r-- 1 710492 2016-02-11 13:37 guice-3.0.jar > > -rw-rw-r-- 1 65012 2016-02-11 13:37 guice-servlet-3.0.jar > > -rw-rw-r-- 1 17035 2016-02-11 13:37 hadoop-annotations-2.6.0.jar > > -rw-rw-r-- 1 67167 2016-02-11 13:37 hadoop-auth-2.6.0.jar > > -rw-rw-r-- 1 3360985 2016-02-11 13:37 hadoop-common-2.6.0.jar > > -rw-rw-r-- 1 7822670 2016-02-11 13:37 hadoop-hdfs-2.6.0.jar > > -rw-rw-r-- 1 664918 2016-02-11 13:37 > hadoop-mapreduce-client-common-2.6.0.jar > > -rw-rw-r-- 1 1509399 2016-02-11 13:37 > hadoop-mapreduce-client-core-2.6.0.jar > > -rw-rw-r-- 1 1870176 2016-02-11 13:37 hadoop-yarn-api-2.6.0.jar > > -rw-rw-r-- 1 127986 2016-02-11 13:37 hadoop-yarn-client-2.6.0.jar > > -rw-rw-r-- 1 1602059 2016-02-11 13:37 hadoop-yarn-common-2.6.0.jar > > -rw-rw-r-- 1 289619 2016-02-11 13:37 > hadoop-yarn-server-common-2.6.0.jar > > -rw-rw-r-- 1 28501 2016-02-11 13:37 > hadoop-yarn-server-web-proxy-2.6.0.jar > > -rw-rw-r-- 1 31212 2016-02-11 13:37 htrace-core-3.0.4.jar > > -rw-rw-r-- 1 433368 2016-02-11 13:37 httpclient-4.2.5.jar > > -rw-rw-r-- 1 227275 2016-02-11 13:37 httpcore-4.2.4.jar > > -rw-rw-r-- 1 232248 2016-02-11 13:37 jackson-core-asl-1.9.13.jar > > -rw-rw-r-- 1 18336 2016-02-11 13:37 jackson-jaxrs-1.9.13.jar > > -rw-rw-r-- 1 780664 2016-02-11 13:37 jackson-mapper-asl-1.9.13.jar > > -rw-rw-r-- 1 27084 2016-02-11 13:37 jackson-xc-1.9.13.jar > > -rw-rw-r-- 1 2497 2016-02-11 13:37 javax.inject-1.jar > > -rw-rw-r-- 1 105134 2016-02-11 13:37 jaxb-api-2.2.2.jar > > -rw-rw-r-- 1 890168 2016-02-11 13:37 jaxb-impl-2.2.3-1.jar > > -rw-rw-r-- 1 130458 2016-02-11 13:37 jersey-client-1.9.jar > > -rw-rw-r-- 1 458739 2016-02-11 13:37 jersey-core-1.9.jar > > -rw-rw-r-- 1 14786 2016-02-11 13:37 jersey-guice-1.9.jar > > -rw-rw-r-- 1 147952 2016-02-11 13:37 jersey-json-1.9.jar > > -rw-rw-r-- 1 81743 2016-02-11 13:37 jettison-1.3.4.jar > > -rw-rw-r-- 1 539912 2016-02-11 13:37 jetty-6.1.26.jar > > -rw-rw-r-- 1 177131 2016-02-11 13:37 jetty-util-6.1.26.jar > > -rw-rw-r-- 1 33031 2016-02-11 13:37 jsr305-3.0.0.jar > > -rw-rw-r-- 1 1045744 2016-02-11 13:37 leveldbjni-all-1.8.jar > > -rw-rw-r-- 1 489884 2016-02-11 13:37 log4j-1.2.17.jar > > -rw-rw-r-- 1 111908 2016-02-11 13:37 metrics-core-3.1.0.jar > > -rw-rw-r-- 1 1303237 2016-02-11 13:37 netty-3.9.2.Final.jar > > -rw-rw-r-- 1 29555 2016-02-11 13:37 paranamer-2.3.jar > > -rw-rw-r-- 1 533455 2016-02-11 13:37 protobuf-java-2.5.0.jar > > -rw-rw-r-- 1 134133 2016-02-11 13:37 servlet-api-2.5-20081211.jar > > -rw-rw-r-- 1 105112 2016-02-11 13:37 servlet-api-2.5.jar > > -rw-rw-r-- 1 32119 2016-02-11 13:37 slf4j-api-1.7.10.jar > > -rw-rw-r-- 1 8866 2016-02-11 13:37 slf4j-log4j12-1.7.10.jar > > -rw-rw-r-- 1 995968 2016-02-11 13:37 snappy-java-1.0.4.1.jar > > -rw-rw-r-- 1 23346 2016-02-11 13:37 stax-api-1.0-2.jar > > -rw-rw-r-- 1 26514 2016-02-11 13:37 stax-api-1.0.1.jar > > -rw-rw-r-- 1 1229125 2016-02-11 13:37 xercesImpl-2.9.1.jar > > -rw-rw-r-- 1 194354 2016-02-11 13:37 xml-apis-1.3.04.jar > > -rw-rw-r-- 1 15010 2016-02-11 13:37 xmlenc-0.52.jar > > -rw-rw-r-- 1 94672 2016-02-11 13:37 xz-1.0.jar > > -rw-rw-r-- 1 792964 2016-02-11 13:37 zookeeper-3.4.6.jar > > > > > > * I am not sure hive compiled against hadoop-2.6.0 (someone else set up > hive and hadoop) > > but where was no problem with hive on m/r. > > this is hive/lib > > -rw-rw-r-- 1 236660 2014-01-30 07:09 ST4-4.0.4.jar > > -rw-rw-r-- 1 4368200 2015-04-30 03:26 accumulo-core-1.6.0.jar > > -rw-rw-r-- 1 102069 2015-04-30 03:26 accumulo-fate-1.6.0.jar > > -rw-rw-r-- 1 57420 2015-04-30 03:26 accumulo-start-1.6.0.jar > > -rw-rw-r-- 1 117409 2015-04-30 03:26 accumulo-trace-1.6.0.jar > > -rw-rw-r-- 1 62983 2014-01-30 07:08 activation-1.1.jar > > -rw-rw-r-- 1 1997485 2014-01-30 07:07 ant-1.9.1.jar > > -rw-rw-r-- 1 18336 2014-01-30 07:07 ant-launcher-1.9.1.jar > > -rw-rw-r-- 1 445288 2014-01-30 07:09 antlr-2.7.7.jar > > -rw-rw-r-- 1 164368 2014-01-30 07:09 antlr-runtime-3.4.jar > > -rw-rw-r-- 1 30359 2015-04-30 03:08 apache-curator-2.6.0.pom > > -rw-rw-r-- 1 448794 2015-04-30 03:08 apache-log4j-extras-1.2.17.jar > > -rw-rw-r-- 1 32693 2014-01-30 11:30 asm-commons-3.1.jar > > -rw-rw-r-- 1 21879 2014-01-30 11:30 asm-tree-3.1.jar > > -rw-rw-r-- 1 400680 2014-01-30 07:08 avro-1.7.5.jar > > -rw-rw-r-- 1 110600 2014-01-30 07:09 bonecp-0.8.0.RELEASE.jar > > -rw-rw-r-- 1 258370 2015-04-30 03:25 > calcite-avatica-1.2.0-incubating.jar > > -rw-rw-r-- 1 3519262 2015-04-30 03:25 > calcite-core-1.2.0-incubating.jar > > -rw-rw-r-- 1 442406 2015-04-30 03:25 > calcite-linq4j-1.2.0-incubating.jar > > -rw-rw-r-- 1 188671 2014-01-30 07:08 commons-beanutils-1.7.0.jar > > -rw-rw-r-- 1 206035 2014-01-30 07:08 > commons-beanutils-core-1.8.0.jar > > -rw-rw-r-- 1 41123 2014-01-30 07:07 commons-cli-1.2.jar > > -rw-rw-r-- 1 58160 2014-01-30 07:07 commons-codec-1.4.jar > > -rw-rw-r-- 1 575389 2014-01-30 07:08 commons-collections-3.2.1.jar > > -rw-rw-r-- 1 30595 2015-04-30 03:25 commons-compiler-2.7.6.jar > > -rw-rw-r-- 1 241367 2014-01-30 07:08 commons-compress-1.4.1.jar > > -rw-rw-r-- 1 298829 2014-01-30 07:08 commons-configuration-1.6.jar > > -rw-rw-r-- 1 160519 2014-05-16 06:37 commons-dbcp-1.4.jar > > -rw-rw-r-- 1 143602 2014-01-30 07:08 commons-digester-1.8.jar > > -rw-rw-r-- 1 279781 2014-01-30 07:07 commons-httpclient-3.0.1.jar > > -rw-rw-r-- 1 185140 2014-01-30 07:08 commons-io-2.4.jar > > -rw-rw-r-- 1 284220 2014-01-30 11:31 commons-lang-2.6.jar > > -rw-rw-r-- 1 62050 2014-01-30 07:07 commons-logging-1.1.3.jar > > -rw-rw-r-- 1 832410 2014-01-30 07:08 commons-math-2.1.jar > > -rw-rw-r-- 1 96221 2014-05-01 09:38 commons-pool-1.5.4.jar > > -rw-rw-r-- 1 415578 2015-04-30 03:26 commons-vfs2-2.0.jar > > -rw-rw-r-- 1 68866 2015-04-30 03:08 curator-client-2.6.0.jar > > -rw-rw-r-- 1 185245 2015-04-30 03:08 curator-framework-2.6.0.jar > > -rw-rw-r-- 1 248171 2015-04-30 03:23 curator-recipes-2.6.0.jar > > -rw-rw-r-- 1 339666 2014-05-13 09:24 datanucleus-api-jdo-3.2.6.jar > > -rw-rw-r-- 1 1890075 2014-05-13 09:24 datanucleus-core-3.2.10.jar > > -rw-rw-r-- 1 1809447 2014-05-13 09:24 datanucleus-rdbms-3.2.9.jar > > -rw-rw-r-- 1 2838580 2015-06-19 18:04 derby-10.10.2.0.jar > > -rw-rw-r-- 1 18482 2015-04-30 03:25 eigenbase-properties-1.1.5.jar > > -rw-rw-r-- 1 12452 2014-01-30 11:30 > geronimo-annotation_1.0_spec-1.1.1.jar > > -rw-rw-r-- 1 30548 2014-01-30 11:30 > geronimo-jaspic_1.0_spec-1.0.jar > > -rw-rw-r-- 1 16030 2014-01-30 11:30 > geronimo-jta_1.1_spec-1.1.1.jar > > -rw-rw-r-- 1 6377448 2014-01-30 07:10 groovy-all-2.1.6.jar > > -rw-rw-r-- 1 2189117 2015-04-30 03:08 guava-14.0.1.jar > > -rw-rw-r-- 1 76643 2014-01-30 07:07 hamcrest-core-1.1.jar > > -rw-rw-r-- 1 121403 2015-06-19 18:05 > hive-accumulo-handler-1.2.1.jar > > -rw-rw-r-- 1 47713 2015-06-19 18:04 hive-ant-1.2.1.jar > > -rw-rw-r-- 1 138361 2015-06-19 18:05 hive-beeline-1.2.1.jar > > -rw-rw-r-- 1 39019 2015-06-19 18:05 hive-cli-1.2.1.jar > > -rw-rw-r-- 1 292290 2015-06-19 18:03 hive-common-1.2.1.jar > > -rw-rw-r-- 1 121668 2015-06-19 18:05 hive-contrib-1.2.1.jar > > -rw-rw-r-- 1 20599030 2015-06-19 18:04 hive-exec-1.2.1.jar > > -rw-rw-r-- 1 115935 2015-06-19 18:05 hive-hbase-handler-1.2.1.jar > > -rw-rw-r-- 1 28091 2015-06-19 18:06 hive-hwi-1.2.1.jar > > -rw-rw-r-- 1 17360142 2015-06-19 18:05 hive-jdbc-1.2.1-standalone.jar > > -rw-rw-r-- 1 100580 2015-06-19 18:05 hive-jdbc-1.2.1.jar > > -rw-rw-r-- 1 5505100 2015-06-19 18:04 hive-metastore-1.2.1.jar > > -rw-rw-r-- 1 916706 2015-06-19 18:03 hive-serde-1.2.1.jar > > -rw-rw-r-- 1 1878543 2015-06-19 18:04 hive-service-1.2.1.jar > > -rw-rw-r-- 1 32390 2015-06-19 18:03 hive-shims-0.20S-1.2.1.jar > > -rw-rw-r-- 1 60070 2015-06-19 18:03 hive-shims-0.23-1.2.1.jar > > -rw-rw-r-- 1 8949 2015-06-19 18:03 hive-shims-1.2.1.jar > > -rw-rw-r-- 1 108914 2015-06-19 18:03 hive-shims-common-1.2.1.jar > > -rw-rw-r-- 1 13065 2015-06-19 18:03 hive-shims-scheduler-1.2.1.jar > > -rw-rw-r-- 1 14530 2015-06-19 18:06 hive-testutils-1.2.1.jar > > -rw-rw-r-- 1 719304 2015-04-30 03:08 httpclient-4.4.jar > > -rw-rw-r-- 1 321639 2015-04-30 03:08 httpcore-4.4.jar > > -rw-rw-r-- 1 1282424 2015-04-30 03:25 ivy-2.4.0.jar > > -rw-rw-r-- 1 611863 2015-04-30 03:25 janino-2.7.6.jar > > -rw-rw-r-- 1 60527 2015-04-30 03:26 jcommander-1.32.jar > > -rw-rw-r-- 1 201124 2014-01-30 07:09 jdo-api-3.0.1.jar > > -rw-rw-r-- 1 1681148 2014-05-13 09:25 jetty-all-7.6.0.v20120127.jar > > -rw-rw-r-- 1 1683027 2014-01-30 11:30 > jetty-all-server-7.6.0.v20120127.jar > > -rw-rw-r-- 1 213854 2015-04-30 03:08 jline-2.12.jar > > -rw-rw-r-- 1 588001 2015-04-30 03:23 joda-time-2.5.jar > > -rw-rw-r-- 1 12131 2014-05-13 09:25 jpam-1.1.jar > > -rw-rw-r-- 1 45944 2014-01-30 07:10 json-20090211.jar > > -rw-rw-r-- 1 33031 2015-04-30 03:23 jsr305-3.0.0.jar > > -rw-rw-r-- 1 15071 2014-01-30 07:09 jta-1.1.jar > > -rw-rw-r-- 1 245039 2015-04-30 03:08 junit-4.11.jar > > -rw-rw-r-- 1 313686 2015-04-30 03:23 libfb303-0.9.2.jar > > -rw-rw-r-- 1 227712 2015-04-30 03:08 libthrift-0.9.2.jar > > -rw-rw-r-- 1 481535 2014-01-30 07:06 log4j-1.2.16.jar > > -rw-rw-r-- 1 447676 2014-01-30 11:30 mail-1.4.1.jar > > -rw-rw-r-- 1 94421 2015-04-30 03:26 maven-scm-api-1.4.jar > > -rw-rw-r-- 1 40066 2015-04-30 03:26 > maven-scm-provider-svn-commons-1.4.jar > > -rw-rw-r-- 1 69858 2015-04-30 03:26 > maven-scm-provider-svnexe-1.4.jar > > -rw-rw-r-- 1 1208356 2015-04-30 03:08 netty-3.7.0.Final.jar > > -rw-rw-r-- 1 19827 2015-04-30 03:23 opencsv-2.3.jar > > -rw-rw-r-- 1 65261 2014-01-30 07:07 oro-2.0.8.jar > > -rw-rw-r-- 1 29555 2014-01-30 07:08 paranamer-2.3.jar > > -rw-rw-r-- 1 2796935 2015-04-30 03:23 > parquet-hadoop-bundle-1.6.0.jar > > -rw-rw-r-- 1 48557 2015-04-30 03:25 > pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar > > drwxrwxr-x 6 4096 2015-09-10 20:14 php > > -rw-rw-r-- 1 250546 2014-01-30 11:29 plexus-utils-1.5.6.jar > > drwxrwxr-x 10 4096 2015-09-10 20:14 py > > -rw-rw-r-- 1 25429 2015-04-30 03:26 regexp-1.3.jar > > -rw-rw-r-- 1 105112 2014-01-30 07:08 servlet-api-2.5.jar > > -rw-rw-r-- 1 1251514 2014-01-30 07:08 snappy-java-1.0.5.jar > > -rw-r--r-- 1 162976273 2015-09-10 20:16 > spark-assembly-1.4.1-hadoop2.6.0.jar > > -rw-rw-r-- 1 26514 2014-01-30 07:08 stax-api-1.0.1.jar > > -rw-rw-r-- 1 148627 2014-01-30 07:09 stringtemplate-3.2.1.jar > > -rw-rw-r-- 1 93210 2015-04-30 03:26 super-csv-2.2.0.jar > > -rw-rw-r-- 1 55953 2014-01-30 11:30 tempus-fugit-1.1.jar > > -rw-rw-r-- 1 392124 2014-01-30 07:07 velocity-1.5.jar > > -rw-rw-r-- 1 94672 2014-01-30 07:08 xz-1.0.jar > > -rw-rw-r-- 1 792964 2015-04-30 03:08 zookeeper-3.4.6.jar > > > > * and hadoop-common-2.6.0.jar have addDeprecations(DeprecationDelta[]) > method. > > <adddeprecations.png> > > > > > > > > 2016-02-12 17:29 GMT+09:00 Hitesh Shah <hit...@apache.org>: > > It seems to me that the Tez AM classpath somehow has a hadoop-common jar > that does not have the Configuration.addDeprecations() api that YARN needs. > > > > For the Tez AM, the classpath is fully constructed based on the tez > tarball ( from HDFS using distributed cache ) and additional jars that Hive > adds ( hive-exec.jar, etc ). It does not use HADOOP_CLASSPATH or anything > else from the cluster nodes. HADOOP_CLASSPATH is only used on the client > node where the hive shell runs. Can you confirm that hive was also compiled > against hadoop-2.6.0 as that might be pulling in a different version of > hadoop-common? > > > > thanks > > — Hitesh > > > > On Feb 12, 2016, at 12:16 AM, no jihun <jees...@gmail.com> wrote: > > > > > Thanks Hitesh Shah. > > > > > > It claims > > > > > > 2016-02-12 14:59:07,388 [ERROR] [main] |app.DAGAppMaster|: Error > starting DAGAppMaster > > > > > > java.lang.NoSuchMethodError: > org.apache.hadoop.conf.Configuration.addDeprecations([Lorg/apache/hadoop/conf/Configuration$DeprecationDelta;)V > > > at > org.apache.hadoop.yarn.conf.YarnConfiguration.addDeprecatedKeys(YarnConfiguration.java:79) > > > at > org.apache.hadoop.yarn.conf.YarnConfiguration.<clinit>(YarnConfiguration.java:73) > > > at > org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2271) > > > > > > I am not sure but according to this thread( > http://grokbase.com/t/cloudera/cdh-user/12765svj61/libjars-and-hadoop-jar-command > ) > > > this perhaps caused by $HADOOP_CLASSPATH problem. > > > > > > But I wander should I copy "tez-dist/target/tez-0.8.2" to all cluster > then export below? > > > export TEZ_JARS=/home1/apps/tez-0.8.2 > > > export TEZ_CONF_DIR=$TEZ_JARS/conf > > > export > HADOOP_CLASSPATH=$TEZ_CONF_DIR:$TEZ_JARS/*:$TEZ_JARS/lib/*:$HADOOP_CLASSPATH > > > > > > I did this only on name nodes. > > > > > > > > > 2016-02-12 16:47 GMT+09:00 Hitesh Shah <hit...@apache.org>: > > > Run the following command: “bin/yarn logs -applicationId > application_1452243782005_0292” . This should give you the logs for > container_1452243782005_0292_02_000001 which may shed more light on why the > Tez ApplicationMaster is failing to launch when triggered via Hive. > > > > > > thanks > > > — Hitesh > > > > > > > > > > > > On Feb 11, 2016, at 10:48 PM, no jihun <jees...@gmail.com> wrote: > > > > > > > Hi all. > > > > > > > > When I execute a query on hive I got an error below.(so do in hive > cli) > > > > no more detailed log found. > > > > > > > > what should I check? > > > > any advice will be appreciated. > > > > > > > > versions > > > > - tez-0.8.2 > > > > - hadoop 2.6.0 > > > > > > > > ----------------------------------- > > > > > > > > hive > set hive.execution.engine=tez; > > > > hive > select count(*) from contents; > > > > > > > > WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. > Please use org.apache.hadoop.log.metrics.EventCounter in all the > log4j.properties files. > > > > > > > > Logging initialized using configuration in > file:/home1/eco/hive/conf/hive-log4j.properties > > > > hive> set hive.execution.engine=tez; > > > > hive> select count(*) from agg_band_contents; > > > > Query ID = irteam_20160212145903_9300f3b2-3942-4423-8586-73d2eaff9e58 > > > > Total jobs = 1 > > > > Launching Job 1 out of 1 > > > > Exception in thread "Thread-10" java.lang.RuntimeException: > org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. > Application application_1452243782005_0292 failed 2 times due to AM > Container for appattempt_1452243782005_0292_000002 exited with exitCode: 1 > > > > For more detailed output, check application tracking page: > http://xstathn003:8088/proxy/application_1452243782005_0292/Then, click > on links to logs of each attempt. > > > > Diagnostics: Exception from container-launch. > > > > Container id: container_1452243782005_0292_02_000001 > > > > Exit code: 1 > > > > Stack trace: ExitCodeException exitCode=1: > > > > at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) > > > > at org.apache.hadoop.util.Shell.run(Shell.java:455) > > > > at > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715) > > > > at > org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211) > > > > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) > > > > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) > > > > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > > > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > > > at java.lang.Thread.run(Thread.java:745) > > > > > > > > > > > > Container exited with a non-zero exit code 1 > > > > Failing this attempt. Failing the application. > > > > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:535) > > > > at > org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:74) > > > > Caused by: org.apache.tez.dag.api.SessionNotRunning: TezSession has > already shutdown. Application application_1452243782005_0292 failed 2 times > due to AM Container for appattempt_1452243782005_0292_000002 exited with > exitCode: 1 > > > > For more detailed output, check application tracking page: > http://xstathn003:8088/proxy/application_1452243782005_0292/Then, click > on links to logs of each attempt. > > > > Diagnostics: Exception from container-launch. > > > > Container id: container_1452243782005_0292_02_000001 > > > > Exit code: 1 > > > > Stack trace: ExitCodeException exitCode=1: > > > > at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) > > > > at org.apache.hadoop.util.Shell.run(Shell.java:455) > > > > at > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715) > > > > at > org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211) > > > > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) > > > > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) > > > > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > > > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > > > at java.lang.Thread.run(Thread.java:745) > > > > > > > > > > > > Container exited with a non-zero exit code 1 > > > > Failing this attempt. Failing the application. > > > > at > org.apache.tez.client.TezClient.waitTillReady(TezClient.java:784) > > > > at > org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:205) > > > > at > org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:116) > > > > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:532) > > > > ... 1 more > > > > Interrupting... Be patient, this might take some time. > > > > > > > > > > > > > > > > > > > > ------------------- > > > > tez example.jar works well. > > > > $ hadoop jar ~/apps/tez-0.8.2/tez-examples-0.8.2.jar > orderedwordcount /tmp/set_webapps.sh /tmp/set_webapps.result > > > > $ ok. > > > > > > > > ------------------- > > > > my setup. > > > > > > > > upload tez to hdfs. > > > > > > > > $ hadoop fs -mkdir /apps > > > > $ hadoop fs -mkdir /apps/tez-0.8.2 > > > > $ hadoop fs -put tez-dist/target/tez-0.8.2.tar.gz /apps/tez-0.8.2/ > > > > > > > > env export > > > > export TEZ_JARS=/home1/irteam/apps/tez-0.8.2 > > > > export TEZ_CONF_DIR=$TEZ_JARS/conf > > > > export > HADOOP_CLASSPATH=$TEZ_CONF_DIR:$TEZ_JARS/*:$TEZ_JARS/lib/*:$HADOOP_CLASSPATH > > > > export HADOOP_USER_CLASSPATH_FIRST=true > > > > $ source ~/.bashrc > > > > > > > > $ mkdir tez-dist/target/tez-0.8.2/conf > > > > $ vi tez-dist/target/tez-0.8.2/conf/tez-site.xml > > > > <?xml version="1.0" encoding="UTF-8"?> > > > > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> > > > > <configuration> > > > > <property> > > > > <name>tez.lib.uris</name> > > > > <value>${fs.defaultFS}/apps/tez-0.8.2/tez-0.8.2.tar.gz</value> > > > > </property> > > > > </configuration> > > > > >
