Re: Sporadic exceptions when checkpointing to S3

2016-07-27 Thread Ufuk Celebi
Hey Gary, your configuration looks good to me. I think that it could be an issue with S3 as you suggest. It might help to decrease the checkpointing interval (if you use case requirements allow for this) in order to have less interaction with S3. In general, your program should still continue as e

Sporadic exceptions when checkpointing to S3

2016-07-27 Thread Gary Yao
Hi all, I am using the filesystem state backend with checkpointing to S3. From the JobManager logs, I can see that it works most of the time, e.g., 2016-07-26 17:49:07,311 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 3 @ 1469555347310 2016-07-26 17