Hi.
Could you share your conf/zeppelin-env.sh and spark-defaults.conf ?
2016-06-28 8:52 GMT+09:00 Jonathan Esterhazy <jonathan.esterh...@gmail.com>:
> I am having trouble using zeppelin in a spark cluster that has spark node
> authentication turned on (e.g. with spark.authenticate=true,
> spark.authenticate.secret=...)
>
> Notebook code that calls built-in spark functions (or other things on
> executor classpath) work fine, but functions defined in the notebook
> (anonymous or named) throw ClassNotFoundExceptions when called from an
> executor.
>
> For example, this code works:
>
> val rdd = sc.textFile("hdfs://my-text-file")
> rdd.take(1).foreach(println)
>
> rdd.saveAsTextFile("hdfs:///my-other-text-file")
>
> but code like this...
>
> rdd.filter(_.contains("my data"))
>
> fails with
>
> Caused by: java.lang.ClassNotFoundException:
> $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$anonfun$1
> at
> org.apache.spark.repl.ExecutorClassLoader.findClass(ExecutorClassLoader.scala:84)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> ...
>
>
> I get the same kind of error if the filter function is defined as a named
> function in the notebook, or as a member of singleton object defined in the
> notebook.
>
> When I look at the executor's log output, I see this error:
>
> 16/06/27 21:36:23 ERROR repl.ExecutorClassLoader: Failed to check
> existence of class
> $line31.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$anonfun$1 on REPL
> class server at https://172.30.54.30:34980
> java.lang.NullPointerException
> at
> org.apache.spark.repl.ExecutorClassLoader.getClassFileInputStreamFromHttpServer(ExecutorClassLoader.scala:113)
> at
> org.apache.spark.repl.ExecutorClassLoader.findClassLocally(ExecutorClassLoader.scala:146)
> at
> org.apache.spark.repl.ExecutorClassLoader.findClass(ExecutorClassLoader.scala:76)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
>
> ...
>
>
> If I disable spark authentication, everything works as expected. I am
> running zeppelin 0.5.6 on spark 1.6.1 with yarn.
>
> Has anyone been able to get zeppelin working with spark authentication?
>
>
>
