can you give the complete REST call you are making to submit the query? From: Xiaoyong Zhu <xiaoy...@microsoft.com<mailto:xiaoy...@microsoft.com>> Reply-To: "user@hive.apache.org<mailto:user@hive.apache.org>" <user@hive.apache.org<mailto:user@hive.apache.org>> Date: Sunday, April 19, 2015 at 8:23 PM To: "user@hive.apache.org<mailto:user@hive.apache.org>" <user@hive.apache.org<mailto:user@hive.apache.org>> Subject: RE: UDF cannot be found when the query is submitted via templeton
No, it doesn't... What surprises me is that for HDInsight (Hadoop on Azure) which is using Azure BLOB storage, using ADD JAR wasb:///test/HiveUDF.jar; CREATE TEMPORARY FUNCTION FindPat as 'HiveUDF.FindPattern' select count(FindPat(columnname)) from table1; would work. However, for my own cluster, ADD JAR hdfs:///test/HiveUDF.jar; CREATE TEMPORARY FUNCTION FindPat as 'HiveUDF.FindPattern' select count(FindPat(columnname)) from table1; does not work... Xiaoyong From: Jason Dere [mailto:jd...@hortonworks.com] Sent: Saturday, April 18, 2015 1:37 AM To: <user@hive.apache.org<mailto:user@hive.apache.org>> Subject: Re: UDF cannot be found when the query is submitted via templeton Does fully qualifying the function name (HiveUDF.FindPattern()) in the query help here? On Apr 17, 2015, at 6:44 AM, Xiaoyong Zhu <xiaoy...@microsoft.com<mailto:xiaoy...@microsoft.com>> wrote: Hi experts I am trying to use an UDF (I have already put that in the metastore using CREATE FUNCTION) as following. select count(FindPattern(s_sitename)) AS testcol from weblogs; However, when I tried to use the UDF from WebHCat (i.e. submit the above command via WebHCat), the job always fails saying Added [/tmp/2cb22c27-72d3-4b41-aea0-655df1192872_resources/HiveUDF.jar] to class path Added resources: [hdfs://PATHTOFOLDER/Portal-Queries/HiveUDF.jar] FAILED: SemanticException [Error 10011]: Line 1:13 Invalid function FindPattern If I execute this command through Hive CLI (through hive -f file or execute it in the interactive shell) the statement above works. From the log I can see the jar file is added but it seems the function cannot be found. Can someone help to share some thoughts on this issue? Btw, the create function statement is as following (changing the hdfs URI to full path does not work either): CREATE FUNCTION FindPattern AS 'HiveUDF.FindPattern' USING JAR'hdfs:///UDFFolder/HiveUDF.jar'<hdfs://UDFFolder/HiveUDF.jar'>; Thanks in advance! Xiaoyong
