Re: Spark SQL Parser error

I have not tried it. But, I guess you need to add your credential in the s3 path. Or, can you copy the jar to your driver node and try again? On Sun, Aug 24, 2014 at 9:35 AM, S Malligarjunan wrote: > Hello Yin, > > Additional note: > In ./bin/spark-shell --jars "s3n:/mybucket/myudf.jar" I got

Re: Spark SQL Parser error

Hello Yin, Additional note: In ./bin/spark-shell --jars "s3n:/mybucket/myudf.jar"  I got the following message in console. Waring skipped external jar..   Thanks and Regards, Sankar S.   On , S Malligarjunan wrote: Hello Yin, I have tried use sc.addJar and hiveContext.sparkContext.addJar

Re: Spark SQL Parser error

Hello Yin, I have tried use sc.addJar and hiveContext.sparkContext.addJar and ./bin/spark-shell --jars option, In all three option when I try to create temporary funtion i get the classNotFoundException. What would be the issue here?   Thanks and Regards, Sankar S.   On Saturday, 23 August 2

Re: Spark SQL Parser error

Hello Sankar, "Add JAR" in SQL is not supported at the moment. We are working on it ( https://issues.apache.org/jira/browse/SPARK-2219). For now, can you try SparkContext.addJar or using "--jars " to launch spark shell? Thanks, Yin On Fri, Aug 22, 2014 at 2:01 PM, S Malligarjunan wrote: > He

Re: Spark SQL Parser error

Hello Yin/All. @Yin - Thanks for helping. I solved the sql parser error. I am getting the following exception now scala> hiveContext.hql("ADD JAR s3n://hadoop.anonymous.com/lib/myudf.jar"); warning: there were 1 deprecation warning(s); re-run with -deprecation for details 14/08/22 17:58:55 INFO

Re: Spark SQL Parser error

Hello Yin, Forgot to mention one thing, the same query works fine in Hive and Shark..   Thanks and Regards, Sankar S.   On , S Malligarjunan wrote: Hello Yin, I have tried  the create external table command as well. I get the same error. Please help me to find the root cause.   Thanks and

Re: Spark SQL Parser error

Hello Yin, I have tried  the create external table command as well. I get the same error. Please help me to find the root cause.   Thanks and Regards, Sankar S.   On Friday, 22 August 2014, 22:43, Yin Huai wrote: Hi Sankar, You need to create an external table in order to specify the loca

Re: Spark SQL Parser error

Hi Sankar, You need to create an external table in order to specify the location of data (i.e. using CREATE EXTERNAL TABLE user1 LOCATION). You can take a look at this page for r