BTW if your looking for protobuf + hive you should look at...
https://github.com/edwardcapriolo/hive-protobuf/ On Sat, Mar 9, 2013 at 12:32 PM, Ramki Palle <ramki.pa...@gmail.com> wrote: > I am not sure whether it works but try this: > > Put your ADD JAR commands into a file and invoke hive with -i file option. > > or > > insert your ADD JAR commands in your $HOME/.hiverc file and start hive. > > > -Ramki. > > > On Fri, Mar 8, 2013 at 11:55 PM, Edward Capriolo <edlinuxg...@gmail.com>wrote: > >> Essentially anything that is part of the InputFormat needs to be in >> auxlib/auxpath. Anything part of a UDF can be added with 'add jar'. >> >> >> On Fri, Mar 8, 2013 at 1:01 PM, Dean Wampler < >> dean.wamp...@thinkbiganalytics.com> wrote: >> >>> --auxpath adds more jars to Hive's classpath before invoking Hive. ADD >>> JARS copies jars around the cluster and adds them to the task classpath, so >>> the jars you add aren't visible to hive itself. Annoying, but... >>> >>> On Fri, Mar 8, 2013 at 11:53 AM, java8964 java8964 <java8...@hotmail.com >>> > wrote: >>> >>>> This is in HIVE-0.9.0 >>>> >>>> hive> list jars; >>>> /nfs_home/common/userlibs/google-collections-1.0.jar >>>> /nfs_home/common/userlibs/elephant-bird-hive-3.0.7.jar >>>> /nfs_home/common/userlibs/protobuf-java-2.3.0.jar >>>> /nfs_home/common/userlibs/elephant-bird-core-3.0.7.jar >>>> file:/usr/lib/hive/lib/hive-builtins-0.9.0-cdh4.1.2.jar >>>> hive> desc table; >>>> java.lang.NoClassDefFoundError: >>>> com/twitter/elephantbird/mapreduce/io/ProtobufConverter >>>> at >>>> com.twitter.elephantbird.hive.serde.ProtobufDeserializer.initialize(ProtobufDeserializer.java:45) >>>> at >>>> org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:203) >>>> at >>>> org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:260) >>>> at >>>> org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253) >>>> at >>>> org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:490) >>>> at >>>> org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:162) >>>> at >>>> org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:930) >>>> at >>>> org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:844) >>>> at >>>> org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:2545) >>>> at >>>> org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:309) >>>> at >>>> org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153) >>>> at >>>> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) >>>> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1331) >>>> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1117) >>>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:950) >>>> at >>>> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258) >>>> at >>>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:215) >>>> at >>>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:406) >>>> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:744) >>>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:607) >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> at >>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >>>> at >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>>> at java.lang.reflect.Method.invoke(Method.java:597) >>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:208) >>>> Caused by: java.lang.ClassNotFoundException: >>>> com.twitter.elephantbird.mapreduce.io.ProtobufConverter >>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >>>> at java.security.AccessController.doPrivileged(Native Method) >>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:307) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:248) >>>> ... 25 more >>>> FAILED: Execution Error, return code -101 from >>>> org.apache.hadoop.hive.ql.exec.DDLTask >>>> hive> exit; >>>> [y130zhan@daca2 userlibs]$ jar tvf >>>> /nfs_home/common/userlibs/elephant-bird-core-3.0.7.jar | grep >>>> ProtobufConverter >>>> 4825 Mon Mar 04 16:50:46 UTC 2013 >>>> com/twitter/elephantbird/mapreduce/io/ProtobufConverter.class >>>> 732 Mon Mar 04 16:50:46 UTC 2013 >>>> com/twitter/elephantbird/mapreduce/io/ProtobufConverter$1.class >>>> >>>> >>>> ------------------------------ >>>> From: vkavul...@outlook.com >>>> To: user@hive.apache.org >>>> Subject: RE: difference between add jar in hive session and hive >>>> --auxpath >>>> Date: Thu, 7 Mar 2013 16:44:41 -0800 >>>> >>>> >>>> If properly done, "add jar <jar-file>" should work the same as passing >>>> the jar with --auxpath. Can you run "list jars;" command from CLI or Hue >>>> and check if you see the jar file. >>>> >>>> ------------------------------ >>>> From: java8...@hotmail.com >>>> To: user@hive.apache.org >>>> Subject: difference between add jar in hive session and hive --auxpath >>>> Date: Thu, 7 Mar 2013 17:47:26 -0500 >>>> >>>> Hi, >>>> >>>> I have a hive table which uses the jar file provided from the >>>> elephant-bird, which is a framework integrated between lzo and google >>>> protobuf data and hadoop/hive. >>>> >>>> If I use the hive command like this: >>>> >>>> hive --auxpath path_to_jars, it works fine to query my table, >>>> >>>> but if I use the add jar after I started the hive session, I will get >>>> ClassNotFoundException in the runtime of my query of the classes in those >>>> jars. >>>> >>>> My questions are: >>>> >>>> 1) What is the different between hive --auxpath and "add jar" in the >>>> hive session? >>>> 2) This problem makes it is hard to access my table in the HUE, as it >>>> only supports "add jar", but not --auxpath option. Any suggestions? >>>> >>>> >>>> Thanks >>>> >>>> Yong >>>> >>> >>> >>> >>> -- >>> *Dean Wampler, Ph.D.* >>> thinkbiganalytics.com >>> +1-312-339-1330 >>> >>> >> >
