Hey! Oozie 4.3.1 and newer releases are compiling with Hadoop 2.6.0. Why is the Hadoop jar you're referring is 2.4.0? Also, what Hadoop version you're compiling Oozie and you're code submitting to Oozie against?
There might be different versions in play on the Oozie classpath and that can cause issues like this. gp On Thu, Jun 7, 2018 at 3:39 PM, Jaboy Mathai <jaboy.mat...@flytxt.com> wrote: > Hi Peter, > > Good Day! > > I managed to build oozie 4.3.1 locally and tried to run job. Oozie starts > up > fine. But I get below exception now in the oozie logs. > > 2018-06-06 00:08:45,862 INFO ActionStartXCommand:520 - > SERVER[svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[] > APP[SAFARICOM_KENYA_CBS_RECHARGE] JOB[0000003-180605235623225- > oozie-hado-W] > ACTION[0000003-180605235623225-oozie-hado-W@GetProcessedFileDetails_ > Recharge_infile] > Start action > [0000003-180605235623225-oozie-hado-W@GetProcessedFileDetails_ > Recharge_infile] > with user-retry state : userRetryCount [0], userRetryMax [0], > userRetryInterval [10] > 2018-06-06 00:08:45,990 ERROR ActionStartXCommand:517 - > SERVER[svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[] > APP[SAFARICOM_KENYA_CBS_RECHARGE] JOB[0000003-180605235623225- > oozie-hado-W] > ACTION[0000003-180605235623225-oozie-hado-W@GetProcessedFileDetails_ > Recharge_infile] > Error, > java.lang.NoSuchMethodError: > org.apache.hadoop.yarn.util.timeline.TimelineUtils. > buildTimelineTokenService(Lorg/apache/hadoop/conf/ > Configuration;)Lorg/apache/hadoop/io/Text; > at > org.apache.hadoop.yarn.client.api.impl.YarnClientImpl. > serviceInit(YarnClientImpl.java:166) > at > org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > at > org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit( > ResourceMgrDelegate.java:102) > at > org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > at > org.apache.hadoop.mapred.ResourceMgrDelegate.<init>( > ResourceMgrDelegate.java:96) > at org.apache.hadoop.mapred.YARNRunner.<init>(YARNRunner.java:112) > at > org.apache.hadoop.mapred.YarnClientProtocolProvider.create( > YarnClientProtocolProvider.java:34) > at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95) > at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82) > at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75) > > > I have checked for the jar file having this method and found that the > class > ' org.apache.hadoop.yarn.util.timeline.TimelineUtils' is present in the > jar > file ' hadoop-yarn-common-2.4.0.jar' but could find the jar for ' > org.apache.hadoop.yarn.util.timeline.TimelineUtils. > buildTimelineTokenService' > , hence don’t know that which would be the dependent jar file here. > > [hadoop@svdt5neonhadoop01 lib]$ > [hadoop@svdt5neonhadoop01 lib]$ egrep -i > org.apache.hadoop.yarn.util.timeline.TimelineUtils * > Binary file hadoop-yarn-common-2.4.0.jar matches > [hadoop@svdt5neonhadoop01 lib]$ > [hadoop@svdt5neonhadoop01 lib]$ egrep -i > org.apache.hadoop.yarn.util.timeline.TimelineUtils. > buildTimelineTokenService > * > [hadoop@svdt5neonhadoop01 lib]$ > [hadoop@svdt5neonhadoop01 lib]$ ls |wc -l > 160 > [hadoop@svdt5neonhadoop01 lib]$ pwd > /usr/local/oozie/lib > [hadoop@svdt5neonhadoop01 lib]$ > > > Please help ! > > Thanks & Regards, > Jaboy Mathai > > > > > -----Original Message----- > From: Peter Cseh [mailto:gezap...@cloudera.com] > Sent: 01 June 2018 15:48 > To: user@oozie.apache.org > Cc: Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam Garg > Subject: Re: Reg: Oozie 4.1.0 having jdbc error > > I'm not aware of a binary distribution of Apache Oozie available to > download. > There are modified versions here and there (e.g. CDH's Oozie can be > installed via rpm: > https://www.cloudera.com/documentation/enterprise/5-14- > x/topics/cdh_ig_cdh5_install.html > , but there are others available as well). > > Peter > > On Fri, Jun 1, 2018 at 11:53 AM, Jaboy Mathai <jaboy.mat...@flytxt.com> > wrote: > > > Dear Peter, > > > > Thanks for your reply. I am using the same Maven 3.5.3 but the java > > version is '1.7.0_67'. Will try to build version 4.3.1 , but I am not > > sure because we don’t have local environment(having RHEL) with > > internet connection as you pointed. That’s why, I requested for a > > build, that may be readily available. > > Let me try building v.4.3.1 and will let you know. > > > > Best Regards > > Jaboy > > > > -----Original Message----- > > From: Peter Cseh [mailto:gezap...@cloudera.com] > > Sent: 01 June 2018 14:49 > > To: user@oozie.apache.org > > Cc: Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam > > Garg > > Subject: Re: Reg: Oozie 4.1.0 having jdbc error > > > > Hi! > > Have you tried building 4.3.1? > > I've managed to build releases 4.3.0, 4.1.0 and 4.3.1. I've even > > cleaned my maven cache in between. > > I'm using Maven 3.5.3 with jdk1.8. > > To resolve the build issue you may compile Oozie locally and copy the > > distribution up to the server to see if it works that way. I suggest > > using Oozie 4.3.1 or 5.0.0. > > > > Based on the error messages during build (timeout) and Oozie startup > > (did not receive any packages) it looks like you're having issues with > > your network. > > > > hope it helps > > gp > > > > On Fri, Jun 1, 2018 at 7:16 AM, Jaboy Mathai <jaboy.mat...@flytxt.com> > > wrote: > > > > > Hi, > > > > > > > > > > > > This is a bit urgent requirement for me, if someone has some idea , > > > please help ! > > > > > > > > > > > > BR, > > > > > > Jaboy > > > > > > > > > > > > *From:* Jaboy Mathai [mailto:jaboy.mat...@flytxt.com] > > > *Sent:* 01 June 2018 10:37 > > > *To:* 'user@oozie.apache.org' > > > *Cc:* Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam > > > Garg > > > *Subject:* RE: Reg: Oozie 4.1.0 having jdbc error > > > > > > > > > > > > + Looping Shivam. > > > > > > > > > > > > *From:* Jaboy Mathai [mailto:jaboy.mat...@flytxt.com > > > <jaboy.mat...@flytxt.com>] > > > *Sent:* 31 May 2018 18:15 > > > *To:* 'user@oozie.apache.org' > > > *Cc:* Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan > > > *Subject:* Reg: Oozie 4.1.0 having jdbc error > > > > > > > > > > > > Hi Team, > > > > > > > > > > > > Need help. > > > > > > > > > > > > Have oozie 4.1.0 installed in RHEL 7.5 in node1. In the same > > > network, we have mariadb 5.5.41 (mariadb.x86_64) in node5(earlier we > > > had version 5.5.56 but we changed that, assuming compatibility issues). > > > Oozie gets started normally , but when I try to start to run the > > > workflow, I get jdbc error in oozie.log > > > > > > > > > > > > *Logs below:* > > > > > > > > > > > > Launcher ERROR, reason: Main class > > > [org.apache.oozie.action.hadoop.JavaMain], > > > main() threw exception, org.springframework.jdbc. > > > CannotGetJdbcConnectionException: Could not get JDBC Connection; > > > nested exception is org.apache.commons.dbcp.SQLNestedException: > > > Cannot create PoolableConnectionFactory (Communications link failure > > > > > > > > > > > > The last packet sent successfully to the server was 0 milliseconds ago. > > > The driver has not received any packets from the server.) > > > > > > 2018-05-31 08:12:23,641 WARN JavaActionExecutor:544 - SERVER[ > > > svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[] > > > APP[SAFARICOM_KENYA_CBS_RECHARGE] > > > JOB[0000001-180531081106985-oozie-hado-W] > > > ACTION[0000001-180531081106985-oozie-hado-W@GetProcessedFileDetails_ > > > Re charge_infile] Launcher exception: org.springframework.jdbc. > > > CannotGetJdbcConnectionException: Could not get JDBC Connection; > > > nested exception is org.apache.commons.dbcp.SQLNestedException: > > > Cannot create PoolableConnectionFactory (Communications link failure > > > > > > > > > > > > The last packet sent successfully to the server was 0 milliseconds ago. > > > The driver has not received any packets from the server.) > > > > > > at > > > org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:58) > > > > > > at org.apache.oozie.action.hadoop.LauncherMain.run( > > > LauncherMain.java:39) > > > > > > at > > > org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:36) > > > > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native > > > Method) > > > > > > at sun.reflect.NativeMethodAccessorImpl.invoke( > > > NativeMethodAccessorImpl.java:57) > > > > > > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > > > DelegatingMethodAccessorImpl.java:43) > > > > > > at java.lang.reflect.Method.invoke(Method.java:606) > > > > > > at org.apache.oozie.action.hadoop.LauncherMapper.map( > > > LauncherMapper.java:226) > > > > > > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) > > > > > > at > > > org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450) > > > > > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) > > > > > > at > > > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) > > > > > > at java.security.AccessController.doPrivileged(Native > > > Method) > > > > > > at javax.security.auth.Subject.doAs(Subject.java:415) > > > > > > at org.apache.hadoop.security.UserGroupInformation.doAs( > > > UserGroupInformation.java:1628) > > > > > > at > > > org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > > > > > > Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: > > > Could not get JDBC Connection; nested exception is > > > org.apache.commons.dbcp.SQLNestedException: > > > Cannot create PoolableConnectionFactory (Communications link failure > > > > > > > > > > > > > > > > > > We have checked the port 3306 for mariadb using ‘nmap’ in linux and > > > have checked the connection hops from node1 to node5 (using > > > traceroute)and found that , there is only a single hop, that is, > > > node1 is directly accessing > > > node5 and vice-versa in the network(no switch IP shown intraceroute > > > output or switch firewalls expected). > > > > > > > > > > > > Checked for iptables and firewalls – no firewalls are running in the > > > cluster. > > > > > > > > > > > > *Please note:* Same job (workflow.xml) is running fine in one of our > > > other cluster with below versions: > > > > > > > > > > > > Oozie 4.1.0 , RHEL 7.3 and database in mariadb 5.5.41. > > > > > > > > > > > > > > > > > > *Attaching the logs having oozie startup and the job run log having > > > the error, with this e-mail.* > > > > > > > > > > > > *Is there any issue with oozie 4.1.0 working in RHEL 7.5 ?* > > > > > > > > > > > > Tried to install oozie 4.3.0 but I am stuck as per below screenshot, > > > when I try to run below command: > > > > > > > > > > > > # ./mkdistro.sh –DskipTests > > > > > > > > > > > > > > > > > > > > > > > > I am able to wget the apache-17.pom from the linux shell prompt but > > > the build process fails. We have added the required proxy to access > > > the internet. I don’t know if any proxy to be added so that the > > > build process comes to know about the internet availability through > > > proxy. > > > > > > > > > > > > Please suggest a solution with the shortest approach possible to > > > solve this. Will appreciate if someone have the oozie 4.3.0 for RHEL > > > 7.5 readily available and could share. > > > > > > > > > > > > Below is the oozie related settings: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > WARN: Use of this script is deprecated; use 'oozied.sh stop' instead > > > > > > > > > > > > Setting OOZIE_HOME: /usr/local/oozie > > > > > > Setting OOZIE_CONFIG: /usr/local/oozie/conf > > > > > > Sourcing: /usr/local/oozie/conf/oozie-env.sh > > > > > > setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m" > > > > > > setting OOZIE_BASE_URL="http://${OOZIE_HTTP_HOSTNAME}:${OOZIE_ > > > HTTP_PORT}/oozie" > > > > > > Setting OOZIE_CONFIG_FILE: oozie-site.xml > > > > > > Setting OOZIE_DATA: /usr/local/oozie/data > > > > > > Setting OOZIE_LOG: /usr/local/oozie/logs > > > > > > Setting OOZIE_LOG4J_FILE: oozie-log4j.properties > > > > > > Setting OOZIE_LOG4J_RELOAD: 10 > > > > > > Setting OOZIE_HTTP_HOSTNAME: svdt5neonhadoop01.safaricom.net > > > > > > Setting OOZIE_HTTP_PORT: 11000 > > > > > > Setting OOZIE_ADMIN_PORT: 11001 > > > > > > Setting OOZIE_HTTPS_PORT: 11443 > > > > > > Using OOZIE_BASE_URL: http://:/oozie > > > > > > Setting CATALINA_BASE: /usr/local/oozie/oozie-server > > > > > > Setting OOZIE_HTTPS_KEYSTORE_FILE: /home/hadoop/.keystore > > > > > > Setting OOZIE_HTTPS_KEYSTORE_PASS: password > > > > > > Setting OOZIE_INSTANCE_ID: svdt5neonhadoop01.safaricom.net > > > > > > Setting CATALINA_OUT: /usr/local/oozie/logs/catalina.out > > > > > > Setting CATALINA_PID: /usr/local/oozie/oozie-server/ > > temp/oozie.pid > > > > > > > > > > > > Using CATALINA_OPTS: -Xmx1024m -Dderby.stream.error.file=/ > > > usr/local/oozie/logs/derby.log > > > > > > Adding to CATALINA_OPTS: -Doozie.home.dir=/usr/local/oozie > > > -Doozie.config.dir=/usr/local/oozie/conf > > > -Doozie.log.dir=/usr/local/oozie/logs > > > -Doozie.data.dir=/usr/local/oozie/data > > > -Doozie.instance.id=svdt5neonh adoop01.safaricom.net > > > -Doozie.config.file=oozie-site.xml > > > -Doozie.log4j.file=oozie-log4j.properties -Doozie.log4j.reload=10 > > > -Doozie.http.hostname=svdt5neonhadoop01.safaricom.net > > > -Doozie.admin.port=11001 -Doozie.http.port=11000 > > > -Doozie.https.port=11443 -Doozie.base.url=http://:/oozie > > > -Doozie.https.keystore.file=/home/hadoop/.keystore > > > -Doozie.https.keystore.pass=password -Djava.library.path= > > > > > > Using CATALINA_BASE: /usr/local/oozie/oozie-server > > > > > > Using CATALINA_HOME: /usr/local/oozie/oozie-server > > > > > > Using CATALINA_TMPDIR: /usr/local/oozie/oozie-server/temp > > > > > > Using JRE_HOME: /usr/java/jdk1.7.0_67 > > > > > > Using CLASSPATH: /usr/local/oozie/oozie-server/bin/bootstrap.jar > > > > > > Using CATALINA_PID: /usr/local/oozie/oozie-server/temp/oozie.pid > > > > > > > > > > > > > > > > > > Thanks & Regards, > > > > > > *Jaboy Mathai* > > > > > > > > > > > > > > > > > -- > > *Peter Cseh *| Software Engineer > > cloudera.com <https://www.cloudera.com> > > > > [image: Cloudera] <https://www.cloudera.com/> > > > > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: > > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: > > Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera> > > ------------------------------ > > > > > > -- > *Peter Cseh *| Software Engineer > cloudera.com <https://www.cloudera.com> > > [image: Cloudera] <https://www.cloudera.com/> > > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera > on LinkedIn] <https://www.linkedin.com/company/cloudera> > ------------------------------ > -- *Peter Cseh *| Software Engineer cloudera.com <https://www.cloudera.com> [image: Cloudera] <https://www.cloudera.com/> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera> ------------------------------
