Those are pretty crappy instructions. Best to build jars with maven or ant or an iDE.
On Thu, Feb 2, 2012 at 8:27 PM, Time Less <timelessn...@gmail.com> wrote: > I sat down with one of our Java engineers who solved it. It actually > appears we're being affected by a slightly-incorrect set of instructions on > the internet. The typical web instructions (duplicated in many places on > the web, such as here > http://www.findnwrite.com/musings/extract-top-n-records-in-each-group-in-hadoophive/) > say to build the .jar like so: > > *$ jar -cf LeoRank.jar /tmp/jar/com/riot/hive/udf/LeoRank.class > * > However, this creates a JAR like so: > > *$ jar -tf LeoRank.jar > META-INF/ > META-INF/MANIFEST.MF > /tmp/jar/com/riot/hive/udf/LeoRank.class > * > Note the class location in the JAR! But we're creating the package as > com.riot.hive.udf, NOT as tmp.jar.com.riot.hive.udf, so it breaks. The > solution is to change the "jar -cf" command like so: > > *$ cd /tmp/jar > $ jar -cf LeoRank.jar ./com/riot/hive/udf/LeoRank.class > $ jar -tf LeoRank.jar > META-INF/ > META-INF/MANIFEST.MF > com/riot/hive/udf/LeoRank.class > * > Note the now-proper class location in the JAR. > > The error is quite generic (read the wide variety of advices given to > people getting this error, from "set your classpath" to "name your function > correctly"). Since the commonly-referenced web instructions for this seem > to lead people astray, hopefully this post will help others in the future. > > -- > Tim Ellis > Data Architect, Riot Games > > > On Thu, Feb 2, 2012 at 5:08 PM, Edward Capriolo <edlinuxg...@gmail.com>wrote: > >> Maybe you are being effected by this? >> >> https://issues.apache.org/jira/browse/HIVE-2635 >> >> >> >> >> On Thu, Feb 2, 2012 at 6:38 PM, Time Less <timelessn...@gmail.com> wrote: >> >>> I'm having a heck of a time the past couple days. Google suggests others >>> have had this same error without resolution since mid-2010. Maybe someone >>> here can shed some light on this? >>> >>> *See the package and method name: >>> *[hdfs@laxhadoop1-012 15:23:14 ~/Tim] :) head LeoRank.java >>> package com.riot.hive.udf; >>> import org.apache.hadoop.hive.ql.exec.UDF; >>> public final class *LeoRank* extends UDF{ >>> >>> *We set the classpath and ensure .jar is there: >>> *[hdfs@laxhadoop1-012 15:24:35 ~/Tim] :) export >>> HADOOP_CLASSPATH=/usr/lib/hive/lib >>> [hdfs@laxhadoop1-012 15:24:44 ~/Tim] :) ls -al >>> $HADOOP_CLASSPATH/LeoRank.jar >>> -rw-rw-r-- 1 hdfs hdfs 884 Feb 2 15:07 /usr/lib/hive/lib/LeoRank.jar >>> >>> *We add the jar: >>> *[hdfs@laxhadoop1-012 15:24:52 ~/Tim] :) hive >>> Hive history file=/tmp/hdfs/hive_job_log_hdfs_201202021524_1137940055.txt >>> hive> add jar LeoRank.jar ; >>> Added LeoRank.jar to class path >>> Added resource: LeoRank.jar >>> >>> *And double-check the .jar is loaded: >>> *hive> list jars ; >>> LeoRank.jar >>> >>> *And yet... fail:* >>> hive> create temporary function LeoRank as 'com.riot.hive.udf.LeoRank' ; >>> FAILED: Execution Error, return code 1 from >>> org.apache.hadoop.hive.ql.exec.FunctionTask >>> >>> *hive.log doesn't seem to shed any additional light on it (or does it?): >>> * >>> [hdfs@laxhadoop1-012 15:31:16 ~/Tim] :) tail -f /tmp/hdfs/hive.log >>> 2012-02-02 15:31:13,087 ERROR ql.Driver >>> (SessionState.java:printError(365)) - FAILED: Execution Error, return code >>> 1 from org.apache.hadoop.hive.ql.exec.FunctionTask >>> >>> There are posts on forums through history where people go through >>> exactly this process, get to exactly this point, then nothing. I'm not sure >>> if they give up on UDFs, or if they have some brain flash around a >>> configuration they forgot to set or what. Any clues for the gamers of the >>> world? >>> >>> -- >>> Tim Ellis >>> Data Architect, Riot Games >>> >>> >> >
