I sat down with one of our Java engineers who solved it. It actually appears we're being affected by a slightly-incorrect set of instructions on the internet. The typical web instructions (duplicated in many places on the web, such as here http://www.findnwrite.com/musings/extract-top-n-records-in-each-group-in-hadoophive/) say to build the .jar like so:
*$ jar -cf LeoRank.jar /tmp/jar/com/riot/hive/udf/LeoRank.class * However, this creates a JAR like so: *$ jar -tf LeoRank.jar META-INF/ META-INF/MANIFEST.MF /tmp/jar/com/riot/hive/udf/LeoRank.class * Note the class location in the JAR! But we're creating the package as com.riot.hive.udf, NOT as tmp.jar.com.riot.hive.udf, so it breaks. The solution is to change the "jar -cf" command like so: *$ cd /tmp/jar $ jar -cf LeoRank.jar ./com/riot/hive/udf/LeoRank.class $ jar -tf LeoRank.jar META-INF/ META-INF/MANIFEST.MF com/riot/hive/udf/LeoRank.class * Note the now-proper class location in the JAR. The error is quite generic (read the wide variety of advices given to people getting this error, from "set your classpath" to "name your function correctly"). Since the commonly-referenced web instructions for this seem to lead people astray, hopefully this post will help others in the future. -- Tim Ellis Data Architect, Riot Games On Thu, Feb 2, 2012 at 5:08 PM, Edward Capriolo <edlinuxg...@gmail.com>wrote: > Maybe you are being effected by this? > > https://issues.apache.org/jira/browse/HIVE-2635 > > > > > On Thu, Feb 2, 2012 at 6:38 PM, Time Less <timelessn...@gmail.com> wrote: > >> I'm having a heck of a time the past couple days. Google suggests others >> have had this same error without resolution since mid-2010. Maybe someone >> here can shed some light on this? >> >> *See the package and method name: >> *[hdfs@laxhadoop1-012 15:23:14 ~/Tim] :) head LeoRank.java >> package com.riot.hive.udf; >> import org.apache.hadoop.hive.ql.exec.UDF; >> public final class *LeoRank* extends UDF{ >> >> *We set the classpath and ensure .jar is there: >> *[hdfs@laxhadoop1-012 15:24:35 ~/Tim] :) export >> HADOOP_CLASSPATH=/usr/lib/hive/lib >> [hdfs@laxhadoop1-012 15:24:44 ~/Tim] :) ls -al >> $HADOOP_CLASSPATH/LeoRank.jar >> -rw-rw-r-- 1 hdfs hdfs 884 Feb 2 15:07 /usr/lib/hive/lib/LeoRank.jar >> >> *We add the jar: >> *[hdfs@laxhadoop1-012 15:24:52 ~/Tim] :) hive >> Hive history file=/tmp/hdfs/hive_job_log_hdfs_201202021524_1137940055.txt >> hive> add jar LeoRank.jar ; >> Added LeoRank.jar to class path >> Added resource: LeoRank.jar >> >> *And double-check the .jar is loaded: >> *hive> list jars ; >> LeoRank.jar >> >> *And yet... fail:* >> hive> create temporary function LeoRank as 'com.riot.hive.udf.LeoRank' ; >> FAILED: Execution Error, return code 1 from >> org.apache.hadoop.hive.ql.exec.FunctionTask >> >> *hive.log doesn't seem to shed any additional light on it (or does it?):* >> [hdfs@laxhadoop1-012 15:31:16 ~/Tim] :) tail -f /tmp/hdfs/hive.log >> 2012-02-02 15:31:13,087 ERROR ql.Driver >> (SessionState.java:printError(365)) - FAILED: Execution Error, return code >> 1 from org.apache.hadoop.hive.ql.exec.FunctionTask >> >> There are posts on forums through history where people go through exactly >> this process, get to exactly this point, then nothing. I'm not sure if they >> give up on UDFs, or if they have some brain flash around a configuration >> they forgot to set or what. Any clues for the gamers of the world? >> >> -- >> Tim Ellis >> Data Architect, Riot Games >> >> >
