Re: Cannot download Jars from S3 due to resource timestamp changed

2019-04-05 Thread Yan Yan
801?focusedCommentId=16807691&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16807691 > > Best, > Tao Yang > > 在 2019年4月5日,上午5:22,Yan Yan 写道: > > Hi, > > I am running issues when trying to move from HDFS to S3 using Flink 1.6. > > I am getting an exception from Hadoop code:

Cannot download Jars from S3 due to resource timestamp changed

2019-04-04 Thread Yan Yan
Hi, I am running issues when trying to move from HDFS to S3 using Flink 1.6. I am getting an exception from Hadoop code: IOException("Resource " + sCopy + " changed on src filesystem (expected " + resource.getTimestamp() + ", was " + sStat.getModificationTime()); Digging into this, I f

Re: HDFS HA issue in Flink 1.4 caused by failing to use fs.hdfs.hadoopconf in Flink conf

2018-10-04 Thread Yan Yan
looping in Stephan and Stefan how > have worked on this the most in the past. Maybe they have some valuable > input. > > Best, > Aljoscha > > > On 4. Oct 2018, at 01:18, Yan Yan wrote: > > Hi, > > We recently bumped to Flink 1.4 from 1.3.2, and found out an issue on

HDFS HA issue in Flink 1.4 caused by failing to use fs.hdfs.hadoopconf in Flink conf

2018-10-03 Thread Yan Yan
Hi, We recently bumped to Flink 1.4 from 1.3.2, and found out an issue on HDFS configuration. We are using *FlinkYarnSessionCli* to start the cluster and submit job. In 1.3.2, we set below Flink properties when using checkpoints: state.backend.fs.checkpointdir = hdfs://nameservice0/.../.. state.