--auxpath adds more jars to Hive's classpath before invoking Hive. ADD JARS copies jars around the cluster and adds them to the task classpath, so the jars you add aren't visible to hive itself. Annoying, but...
On Fri, Mar 8, 2013 at 11:53 AM, java8964 java8964 <java8...@hotmail.com>wrote: > This is in HIVE-0.9.0 > > hive> list jars; > /nfs_home/common/userlibs/google-collections-1.0.jar > /nfs_home/common/userlibs/elephant-bird-hive-3.0.7.jar > /nfs_home/common/userlibs/protobuf-java-2.3.0.jar > /nfs_home/common/userlibs/elephant-bird-core-3.0.7.jar > file:/usr/lib/hive/lib/hive-builtins-0.9.0-cdh4.1.2.jar > hive> desc table; > java.lang.NoClassDefFoundError: > com/twitter/elephantbird/mapreduce/io/ProtobufConverter > at > com.twitter.elephantbird.hive.serde.ProtobufDeserializer.initialize(ProtobufDeserializer.java:45) > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:203) > at > org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:260) > at > org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253) > at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:490) > at > org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:162) > at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:930) > at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:844) > at > org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:2545) > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:309) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1331) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1117) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:950) > at > org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258) > at > org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:215) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:406) > at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:744) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:607) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:208) > Caused by: java.lang.ClassNotFoundException: > com.twitter.elephantbird.mapreduce.io.ProtobufConverter > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > at java.lang.ClassLoader.loadClass(ClassLoader.java:307) > at java.lang.ClassLoader.loadClass(ClassLoader.java:248) > ... 25 more > FAILED: Execution Error, return code -101 from > org.apache.hadoop.hive.ql.exec.DDLTask > hive> exit; > [y130zhan@daca2 userlibs]$ jar tvf > /nfs_home/common/userlibs/elephant-bird-core-3.0.7.jar | grep > ProtobufConverter > 4825 Mon Mar 04 16:50:46 UTC 2013 > com/twitter/elephantbird/mapreduce/io/ProtobufConverter.class > 732 Mon Mar 04 16:50:46 UTC 2013 > com/twitter/elephantbird/mapreduce/io/ProtobufConverter$1.class > > > ------------------------------ > From: vkavul...@outlook.com > To: user@hive.apache.org > Subject: RE: difference between add jar in hive session and hive --auxpath > Date: Thu, 7 Mar 2013 16:44:41 -0800 > > > If properly done, "add jar <jar-file>" should work the same as passing the > jar with --auxpath. Can you run "list jars;" command from CLI or Hue and > check if you see the jar file. > > ------------------------------ > From: java8...@hotmail.com > To: user@hive.apache.org > Subject: difference between add jar in hive session and hive --auxpath > Date: Thu, 7 Mar 2013 17:47:26 -0500 > > Hi, > > I have a hive table which uses the jar file provided from the > elephant-bird, which is a framework integrated between lzo and google > protobuf data and hadoop/hive. > > If I use the hive command like this: > > hive --auxpath path_to_jars, it works fine to query my table, > > but if I use the add jar after I started the hive session, I will get > ClassNotFoundException in the runtime of my query of the classes in those > jars. > > My questions are: > > 1) What is the different between hive --auxpath and "add jar" in the hive > session? > 2) This problem makes it is hard to access my table in the HUE, as it only > supports "add jar", but not --auxpath option. Any suggestions? > > > Thanks > > Yong > -- *Dean Wampler, Ph.D.* thinkbiganalytics.com +1-312-339-1330
