Re: Exception while submitting jobs through Yarn

2018-06-20 Thread Till Rohrmann
>> user forum. >> >> Cheers >> >> Original message >> From: Garvit Sharma >> Date: 6/19/18 3:34 AM (GMT-08:00) >> To: trohrm...@apache.org >> Cc: Amit Jain , Chesnay Schepler , >> Ted Yu , user@flink.apache.org >> Su

Re: Exception while submitting jobs through Yarn

2018-06-20 Thread Garvit Sharma
epler , > Ted Yu , user@flink.apache.org > Subject: Re: Exception while submitting jobs through Yarn > > Any help on this? > > On Mon, Jun 18, 2018 at 11:31 PM Garvit Sharma > wrote: > >> Yes, it is. >> >> On Mon, Jun 18, 2018 at 7:54 PM Till Rohrmann

Re: Exception while submitting jobs through Yarn

2018-06-19 Thread Ted Yu
Exception while submitting jobs through Yarn Any help on this? On Mon, Jun 18, 2018 at 11:31 PM Garvit Sharma wrote: Yes, it is.  On Mon, Jun 18, 2018 at 7:54 PM Till Rohrmann wrote: Is `/usr/hdp/2.6.3.0-235/hadoop/client/xercesImpl.jar` a link to `/usr/hdp/2.6.3.0-235/hadoop/client/xercesImpl

Re: Exception while submitting jobs through Yarn

2018-06-19 Thread Garvit Sharma
t;>> Maximum >>>>>>>>>> heap size: 6647 MiBytes >>>>>>>>>> 2018-06-17 19:01:31,663 INFO >>>>>>>>>> org.apache.flink.yarn.YarnTaskExecutorRunner - >>>>>>>>>> JAVA_HOME: /usr

Re: Exception while submitting jobs through Yarn

2018-06-18 Thread Garvit Sharma
t;>>>> org.apache.flink.yarn.YarnTaskExecutorRunner - JVM >>>>>>>>> Options: >>>>>>>>> 2018-06-17 19:01:31,665 INFO >>>>>>>>> org.apache.flink.yarn.YarnTaskExecutorRunner - >>>>

Re: Exception while submitting jobs through Yarn

2018-06-18 Thread Till Rohrmann
:01:31,665 INFO >>>>>>>> org.apache.flink.yarn.YarnTaskExecutorRunner - >>>>>>>> -XX:MaxDirectMemorySize=4072m >>>>>>>> 2018-06-17 19:01:31,665 INFO >>>>>>>> org.apache.fl

Re: Exception while submitting jobs through Yarn

2018-06-18 Thread Garvit Sharma
gt;>>>> 2018-06-17 19:01:31,665 INFO >>>>>>> org.apache.flink.yarn.YarnTaskExecutorRunner - >>>>>>> -Dlogback.configurationFile=file:./logback.xml >>>>>>> 2018-06-17 19:01:31,665 INFO >>>>>>> org.apache.flin

Re: Exception while submitting jobs through Yarn

2018-06-18 Thread Till Rohrmann
he.flink.yarn.YarnTaskExecutorRunner - >>>>>> --configDir >>>>>> 2018-06-17 19:01:31,665 INFO >>>>>> org.apache.flink.yarn.YarnTaskExecutorRunner - . >>>>>> *2018-06-17 19:01:31,666 INFO

Re: Exception while submitting jobs through Yarn

2018-06-18 Thread Garvit Sharma
b/flink-python_2.11-1.5.0.jar:lib/flink-shaded-hadoop2-uber-1.5.0.jar:lib/flink-shaded-include-yarn-0.9.1.jar:lib/guava-18.0.jar:lib/log4j-1.2.17.jar:lib/slf4j-log4j12-1.7.7.jar:log4j.properties:logback.xml:flink.jar:flink-conf.yaml::/etc/hadoop/conf:/usr/lib/hadoop/hadoop-common-2.8.3-amzn-0.jar:/usr/lib/h

Re: Exception while submitting jobs through Yarn

2018-06-18 Thread Till Rohrmann
doop-azure-datalake-2.8.3-amzn-0.jar.....* >>>> >>>> -- >>>> Thanks, >>>> Amit >>>> >>>> On Mon, Jun 18, 2018 at 2:00 PM, Garvit Sharma >>>> wrote: >>>>

Re: Exception while submitting jobs through Yarn

2018-06-18 Thread Garvit Sharma
t;>> Thanks, >>> Amit >>> >>> On Mon, Jun 18, 2018 at 2:00 PM, Garvit Sharma >>> wrote: >>> >>>> Hi, >>>> >>>> Please refer to my previous mail for complete logs. >>>> >>>> Thanks, >>>> >

Re: Exception while submitting jobs through Yarn

2018-06-18 Thread Till Rohrmann
file with us. >>>> >>>> Cheers, >>>> Till >>>> >>>> On Sat, Jun 16, 2018 at 5:22 PM Ted Yu wrote: >>>> >>>>> The error for core-default.xml is interesting. >>>>> >>>>> Flink doesn't have

Re: Exception while submitting jobs through Yarn

2018-06-18 Thread Garvit Sharma
>>> >>>> The error for core-default.xml is interesting. >>>> >>>> Flink doesn't have this file. Probably it came with Yarn. Please check >>>> the hadoop version Flink was built with versus the hadoop version in your >>>> cluster. >>>

Re: Exception while submitting jobs through Yarn

2018-06-18 Thread Amit Jain
>> the hadoop version Flink was built with versus the hadoop version in your >>> cluster. >>> >>> Thanks >>> >>> Original message >>> From: Garvit Sharma >>> Date: 6/16/18 7:23 AM (GMT-08:00) >>> To: trohrm...@apache

Re: Exception while submitting jobs through Yarn

2018-06-18 Thread Garvit Sharma
sage >> From: Garvit Sharma >> Date: 6/16/18 7:23 AM (GMT-08:00) >> To: trohrm...@apache.org >> Cc: Chesnay Schepler , user@flink.apache.org >> Subject: Re: Exception while submitting jobs through Yarn >> >> I am not able to figure out,

Re: Exception while submitting jobs through Yarn

2018-06-18 Thread Till Rohrmann
t with versus the hadoop version in your > cluster. > > Thanks > > Original message > From: Garvit Sharma > Date: 6/16/18 7:23 AM (GMT-08:00) > To: trohrm...@apache.org > Cc: Chesnay Schepler , user@flink.apache.org > Subject: Re: Exception while su

Re: Exception while submitting jobs through Yarn

2018-06-16 Thread Ted Yu
MT-08:00) To: trohrm...@apache.org Cc: Chesnay Schepler , user@flink.apache.org Subject: Re: Exception while submitting jobs through Yarn I am not able to figure out, got stuck badly in this since last 1 week. Any little help would be appreciated. 2018-06-16 19:25:10,523

Re: Exception while submitting jobs through Yarn

2018-06-16 Thread Garvit Sharma
I am not able to figure out, got stuck badly in this since last 1 week. Any little help would be appreciated. 2018-06-16 19:25:10,523 DEBUG org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator - Parallelism set: 1 for 8 2018-06-16 19:25:10,578 DEBUG org.apache.flink.streaming.api.gra

Re: Exception while submitting jobs through Yarn

2018-06-15 Thread Till Rohrmann
Hmm could you maybe share the client logs with us. Cheers, Till On Fri, Jun 15, 2018 at 4:54 PM Garvit Sharma wrote: > Yes, I did. > > On Fri, Jun 15, 2018 at 6:17 PM Till Rohrmann > wrote: > >> Hi Garvit, >> >> have you exported the HADOOP_CLASSPATH as described in the release notes >> [1]? >

Re: Exception while submitting jobs through Yarn

2018-06-15 Thread Garvit Sharma
Yes, I did. On Fri, Jun 15, 2018 at 6:17 PM Till Rohrmann wrote: > Hi Garvit, > > have you exported the HADOOP_CLASSPATH as described in the release notes > [1]? > > [1] > https://ci.apache.org/projects/flink/flink-docs-release-1.5/release-notes/flink-1.5.html#hadoop-classpath-discovery > > Chee

Re: Exception while submitting jobs through Yarn

2018-06-15 Thread Till Rohrmann
Hi Garvit, have you exported the HADOOP_CLASSPATH as described in the release notes [1]? [1] https://ci.apache.org/projects/flink/flink-docs-release-1.5/release-notes/flink-1.5.html#hadoop-classpath-discovery Cheers, Till On Fri, Jun 15, 2018 at 2:22 PM Garvit Sharma wrote: > Does someone has

Re: Exception while submitting jobs through Yarn

2018-06-15 Thread Garvit Sharma
Does someone has any idea how to get rid if the above parse exception while submitting flink job to Yarn. Already searched on the internet, could not find any solution to it. Please help. On Fri, Jun 15, 2018 at 9:15 AM Garvit Sharma wrote: > Thanks Chesnay, Now it is connecting to the Resourc

Re: Exception while submitting jobs through Yarn

2018-06-14 Thread Garvit Sharma
Thanks Chesnay, Now it is connecting to the Resource Manager but I am getting the below exception : 2018-06-15 09:12:44,812 FATAL org.apache.hadoop.conf.Configuration - error parsing conf core-default.xml javax.xml.parsers.ParserConfigurationException: Feature ' http://apache.org/

Re: Exception while submitting jobs through Yarn

2018-06-14 Thread Chesnay Schepler
My gut feeling is that these classes must be present in jars in the /lib directory. I don't think you can supply these with the submitted jar. For a simple test, put your jar into the /lib folder before submitting it. On 14.06.2018 06:56, Garvit Sharma wrote: Can someone please tell why am I fa

Re: Exception while submitting jobs through Yarn

2018-06-13 Thread Garvit Sharma
Can someone please tell why am I facing this? On Wed, Jun 13, 2018 at 10:33 PM Garvit Sharma wrote: > Hi, > > I am using *flink-1.5.0-bin-hadoop27-scala_2.11 *to submit jobs through > Yarn, but I am getting the below exception : > > java.lang.NoClassDefFoundError: > com/sun/jersey/core/util/Feat