Hi,
I am trying to run a Spark action (Oozie 4.2.0) and everything works fine
if my action is run in local mode. As soon as I switch to cluster mode with
yarn as master I get a NullPointerException from Oozie:
JOB[0000724-160602113052690-oozie-oozi-W]
ACTION[0000724-160602113052690-oozie-oozi-W@submit-spark-job] Error
starting action [submit-spark-job]. ErrorType [ERROR], ErrorCode
[NullPointerException], Message [NullPointerException: null]
org.apache.oozie.action.ActionExecutorException: NullPointerException: null
at
org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:445)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1139)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1293)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:250)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:64)
at org.apache.oozie.command.XCommand.call(XCommand.java:286)
at org.apache.oozie.command.XCommand.call(XCommand.java:356)
at
org.apache.oozie.command.wf.SignalXCommand.execute(SignalXCommand.java:434)
at
org.apache.oozie.command.wf.SignalXCommand.execute(SignalXCommand.java:76)
at org.apache.oozie.command.XCommand.call(XCommand.java:286)
at org.apache.oozie.command.XCommand.call(XCommand.java:356)
at
org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:280)
at
org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:61)
at org.apache.oozie.command.XCommand.call(XCommand.java:286)
at org.apache.oozie.command.XCommand.call(XCommand.java:356)
at
org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:244)
at
org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:56)
at org.apache.oozie.command.XCommand.call(XCommand.java:286)
at
org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at
org.apache.oozie.action.hadoop.SparkActionExecutor.setupActionConf(SparkActionExecutor.java:79)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1042)
... 20 more
I am wondering if this is somehow related to:
https://issues.apache.org/jira/browse/SPARK-7706
In any case, what is the cause? is there a workaround?
Thanks for your help,
Pierre
