Ok, seems like something wrong when you try to use deps. I was able run
simple spark job w/o third party dependecies.
Zeppelin always throw NPE when you try to use local files using %spark.dep
or spark interpreter conf (there is an option to set local file).
Did anyone make it work?
2017-06-26 21:31 GMT+02:00 Serega Sheypak <serega.shey...@gmail.com>:
> Hi, I'm getting strange NPE w/o any obvious reason.
>
> My notebook contains two paragraphs:
>
>
> res0: org.apache.zeppelin.dep.Dependency = org.apache.zeppelin.dep.
> Dependency@6ce5acd
>
> %spark.dep z.load("some-local-jar.jar")
> and
>
> import com.SuperClass
> // bla-bla
>
> val features = sc.sequenceFile[NullWritable, BytesWritable](path)
> // bla-bla
> }
>
> println(features)
>
> Can you give me any input, how can I debug it?
>
> exception in UI is:
> java.lang.NullPointerException at org.apache.zeppelin.spark.
> SparkInterpreter.open(SparkInterpreter.java:887) at org.apache.zeppelin.
> interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:70) at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$
> InterpretJob.jobRun(RemoteInterpreterServer.java:489) at
> org.apache.zeppelin.scheduler.Job.run(Job.java:181) at
> org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:47
>
> exception in interpreter log is:
>
> ERROR [2017-06-26 08:48:15,846] ({pool-2-thread-2}
> Utils.java[invokeMethod]:41) -
>
> java.lang.reflect.InvocationTargetException
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:57)
>
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:39)
>
> at org.apache.zeppelin.spark.SparkInterpreter.interpret(
> SparkInterpreter.java:1008)
>
> at org.apache.zeppelin.spark.SparkInterpreter.open(
> SparkInterpreter.java:880)
>
> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(
> LazyOpenInterpreter.java:70)
>
> at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$
> InterpretJob.jobRun(RemoteInterpreterServer.java:489)
>
> at org.apache.zeppelin.scheduler.Job.run(Job.java:181)
>
> at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(
> FIFOScheduler.java:139)
>
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>
> at java.util.concurrent.ScheduledThreadPoolExecutor$
> ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>
> at java.util.concurrent.ScheduledThreadPoolExecutor$
> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1145)
>
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
> Caused by: java.lang.NullPointerException
>
> at scala.reflect.internal.SymbolTable.exitingPhase(SymbolTable.scala:256)
>
> at scala.tools.nsc.interpreter.IMain$Request.x$20$lzycompute(
> IMain.scala:896)
>
> at scala.tools.nsc.interpreter.IMain$Request.x$20(IMain.scala:895)
>
> at scala.tools.nsc.interpreter.IMain$Request.headerPreamble$
> lzycompute(IMain.scala:895)
>
> at scala.tools.nsc.interpreter.IMain$Request.headerPreamble(
> IMain.scala:895)
>
> at scala.tools.nsc.interpreter.IMain$Request$Wrapper.
> preamble(IMain.scala:918)
>
> at scala.tools.nsc.interpreter.IMain$CodeAssembler$$anonfun$
> apply$23.apply(IMain.scala:1337)
>
> at scala.tools.nsc.interpreter.IMain$CodeAssembler$$anonfun$
> apply$23.apply(IMain.scala:1336)
>
> at scala.tools.nsc.util.package$.stringFromWriter(package.scala:64)
>
> at scala.tools.nsc.interpreter.IMain$CodeAssembler$class.
> apply(IMain.scala:1336)
>
> at scala.tools.nsc.interpreter.IMain$Request$Wrapper.apply(
> IMain.scala:908)
>
> at scala.tools.nsc.interpreter.IMain$Request.compile$
> lzycompute(IMain.scala:1002)
>
> at scala.tools.nsc.interpreter.IMain$Request.compile(IMain.scala:997)
>
> at scala.tools.nsc.interpreter.IMain.compile(IMain.scala:579)
>
> at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:567)
>
> at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
>
> ... 18 more
>
> ERROR [2017-06-26 08:48:15,847] ({pool-2-thread-2} Job.java[run]:186) -
> Job failed
>
> java.lang.NullPointerException
>
> at org.apache.zeppelin.spark.SparkInterpreter.open(
> SparkInterpreter.java:887)
>
> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(
> LazyOpenInterpreter.java:70)
>
> at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$
> InterpretJob.jobRun(RemoteInterpreterServer.java:489)
>
> at org.apache.zeppelin.scheduler.Job.run(Job.java:181)
>
> at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(
> FIFOScheduler.java:139)
>
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>
> at java.util.concurrent.ScheduledThreadPoolExecutor$
> ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>
> at java.util.concurrent.ScheduledThreadPoolExecutor$
> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1145)
>
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
>
>
> Server log:
>
> INFO [2017-06-26 08:47:53,248] ({pool-2-thread-12}
> RemoteInterpreterManagedProcess.java[start]:132) - Run interpreter
> process [/home/ssheypak/zeppelin-0.8.0-SNAPSHOT/bin/interpreter.sh, -d,
> /home/ssheypak/zeppelin-0.8.0-SNAPSHOT/interpreter/spark, -p, 45601, -l,
> /home/ssheypak/zeppelin-0.8.0-SNAPSHOT/local-repo/2CNTZMQPW, -g, spark]
>
> INFO [2017-06-26 08:47:54,255] ({pool-2-thread-12}
> RemoteInterpreter.java[init]:222) - Create remote interpreter
> org.apache.zeppelin.spark.SparkInterpreter
>
> INFO [2017-06-26 08:47:54,387] ({pool-2-thread-12} RemoteInterpreter.java[
> pushAngularObjectRegistryToRemote]:552) - Push local angular object
> registry from ZeppelinServer to remote interpreter group
> 2CNTZMQPW:shared_process
>
> INFO [2017-06-26 08:47:54,405] ({pool-2-thread-12}
> RemoteInterpreter.java[init]:222) - Create remote interpreter
> org.apache.zeppelin.spark.SparkSqlInterpreter
>
> INFO [2017-06-26 08:47:54,409] ({pool-2-thread-12}
> RemoteInterpreter.java[init]:222) - Create remote interpreter
> org.apache.zeppelin.spark.DepInterpreter
>
> INFO [2017-06-26 08:47:54,411] ({pool-2-thread-12}
> RemoteInterpreter.java[init]:222) - Create remote interpreter
> org.apache.zeppelin.spark.PySparkInterpreter
>
> ERROR [2017-06-26 08:47:54,424] ({qtp1537491187-239}
> NotebookServer.java[onMessage]:380) - Can't handle message
>
> java.lang.NullPointerException
>
> at org.apache.zeppelin.socket.NotebookServer.
> addNewParagraphIfLastParagraphIsExecuted(NotebookServer.java:1713)
>
> at org.apache.zeppelin.socket.NotebookServer.
> persistAndExecuteSingleParagraph(NotebookServer.java:1741)
>
> at org.apache.zeppelin.socket.NotebookServer.runAllParagraphs(
> NotebookServer.java:1641)
>
> at org.apache.zeppelin.socket.NotebookServer.onMessage(
> NotebookServer.java:291)
>
> at org.apache.zeppelin.socket.NotebookSocket.onWebSocketText(
> NotebookSocket.java:59)
>
> at org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.
> onTextMessage(JettyListenerEventDriver.java:128)
>
> at org.eclipse.jetty.websocket.common.message.SimpleTextMessage.
> messageComplete(SimpleTextMessage.java:69)
>
> at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.
> appendMessage(AbstractEventDriver.java:65)
>
> at org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.
> onTextFrame(JettyListenerEventDriver.java:122)
>
> at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.
> incomingFrame(AbstractEventDriver.java:161)
>
> at org.eclipse.jetty.websocket.common.WebSocketSession.incomingFrame(
> WebSocketSession.java:309)
>
> at org.eclipse.jetty.websocket.common.extensions.
> ExtensionStack.incomingFrame(ExtensionStack.java:214)
>
> at org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:220)
>
> at org.eclipse.jetty.websocket.common.Parser.parse(Parser.java:258)
>
> at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.
> readParse(AbstractWebSocketConnection.java:632)
>
> at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.
> onFillable(AbstractWebSocketConnection.java:480)
>
> at org.eclipse.jetty.io.AbstractConnection$2.run(
> AbstractConnection.java:544)
>
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
> QueuedThreadPool.java:635)
>
> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(
> QueuedThreadPool.java:555)
>
> at java.lang.Thread.run(Thread.java:745)
>
> INFO [2017-06-26 08:47:54,924] ({pool-2-thread-10}
> SchedulerFactory.java[jobStarted]:131)
> - Job paragraph_1498490257732_1130121833 started by scheduler
> org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_
> session426301004
>
> INFO [2017-06-26 08:47:54,925] ({pool-2-thread-10}
> Paragraph.java[jobRun]:363) - run paragraph 20170626-081737_1275486301
> using null org.apache.zeppelin.interpreter.LazyOpenInterpreter@55b139c6
>
> INFO [2017-06-26 08:48:00,542] ({pool-2-thread-12}
> NotebookServer.java[afterStatusChange]:2160)
> - Job 20170626-081602_215470268 is finished successfully, status: FINISHED
>
> INFO [2017-06-26 08:48:00,567] ({pool-2-thread-12}
> SchedulerFactory.java[jobFinished]:137)
> - Job paragraph_1498490162079_2075880602 <(207)%20588-0602> finished by
> scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_
> session426301004
>
> WARN [2017-06-26 08:48:15,876] ({pool-2-thread-10}
> NotebookServer.java[afterStatusChange]:2162)
> - Job 20170626-081737_1275486301 is finished, status: ERROR, exception:
> null, result: %text java.lang.NullPointerException
>
> at org.apache.zeppelin.spark.SparkInterpreter.open(
> SparkInterpreter.java:887)
>
> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(
> LazyOpenInterpreter.java:70)
>
> at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$
> InterpretJob.jobRun(RemoteInterpreterServer.java:489)
>
> at org.apache.zeppelin.scheduler.Job.run(Job.java:181)
>
> at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(
> FIFOScheduler.java:139)
>
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>
> at java.util.concurrent.ScheduledThreadPoolExecutor$
> ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>
> at java.util.concurrent.ScheduledThreadPoolExecutor$
> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1145)
>
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
>
>
>
>
>
