Hi, I am facing a strange issue on Oozie 5.0.
I have an Ozzie application that works fine in the workflow mode.
Here is the only 2 examples line of my oozie properties file:
oozie.wf.application.path=${appBaseDir}/workflow/workflow.xml
#oozie.coord.application.path=${appBaseDir}/workflow/coordinator.xml
And here is the content of the coordinator.xml file:
<coordinator-app name="GoogleAnalytics" start="2018-09-06T23:25Z"
end="2100-12-31T23:00Z" frequency="${coord:days(1)}"
timezone="UTC"
xmlns="uri:oozie:coordinator:0.4">
<action>
<workflow>
<app-path>${appBaseDir}/workflow/workflow.xml</app-path>
<configuration>
<property>
<name>nameNode</name>
<value>${nameNode}</value>
</property>
<property>
<name>jobTracker</name>
<value>${jobTracker}</value>
</property>
</configuration>
</workflow>
</action>
</coordinator-app>
Now, this whole thing works on AWS EMR with Oozie 4.3. But with the new version
of AWS EMR comes with Oozie 5.0.0, I am facing the following strange error.
1) If I leave the properties file as above to running in the workflow mode by
"oozie job -run -config googleanalytics_aws_qa.properties file, everything
works under oozie 5.0
2) If I comment out the "wf" line above and uncomment out the "coord" line, I
got the following error:
[hadoop@ip-10-0-60-152 oozieJobs]$ oozie job -run -config
googleanalytics_aws_qa.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/lib/oozie/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/lib/oozie/lib/slf4j-simple-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Error: E0307 : E0307: Runtime error [App directory [null] does not exist and
app definition cannot be created because of missing config value
[oozie.jobs.api.generated.xml]]
And here is the stack trace in the oozie.log:
2019-04-01 22:03:55,806 WARN V1JobsServlet:523 -
SERVER[ip-10-0-60-152.ec2.internal] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
ACTION[-] URL[POST
http://ip-10-0-60-152.ec2.internal:11000/oozie/v2/jobs?action=start&user=hadoop]
error[E0307], E0307: Runtime error [App directory [null] does not exist and
app definition cannot be created because of missing config value
[oozie.jobs.api.generated.xml]]
org.apache.oozie.servlet.XServletException: E0307: Runtime error [App directory
[null] does not exist and app definition cannot be created because of missing
config value [oozie.jobs.api.generated.xml]]
at
org.apache.oozie.servlet.V1JobsServlet.checkAndWriteApplicationXMLToHDFS(V1JobsServlet.java:175)
at
org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:111)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at
org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:305)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
at org.apache.oozie.servlet.AuthFilter$2.doFilter(AuthFilter.java:171)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:644)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:592)
at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:176)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.apache.oozie.servlet.HostnameFilter.doFilter(HostnameFilter.java:86)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
I checked again and again, and I have no idea what this "[App directory [null]"
coming from.
* I keep using the old XSD as workflow 0.3 and coordinator:0.5 under oozie
5.0, that should be supported right?
* I didn't see any issue of the application path in the properties file,
and as it works under "workflow" mode, so there should be no issue.
* Since this error ONLY happens in the coordinator mode, and the only issue
COULD come from the coordinator.xml. As I pasted above, it is so simple, and I
really not sure where the issue comes from.
* I even use the XSD from
https://oozie.apache.org/docs/5.0.0/CoordinatorFunctionalSpec.html#Oozie_Coordinator_Schema_0.4
to validate my coordinator.xml, and it is valid.
* So my questions are:
* Where is this App directory [null] coming from?
* The root reason for failure is due to AppDirectory is NULL, or
definition cannot be created because of missing config value. If later, what
kind of config value is missing we are talking about here?
* If it is due to AppDirectory is NULL, what could cause it ONLY happens
in Coord mode, but work fine in Workflow mode?
Thanks
Yong
