Re: Re: Checkpoints aborted - Job is not in state RUNNING but FINISHED

2021-11-29 Thread Yun Gao
Hi Jonas, For the previos versions, the checkpoint would be aborted as long as any task get finished, no matter if they are from the save vertex. And for the Kinesis, sorry I do not find an environment to do a test, but if the task would indeed finished if there are no shards, I think it would in

Re: Checkpoints aborted - Job is not in state RUNNING but FINISHED

2021-11-26 Thread jonas eyob
Hi Yun, thanks for the quick reply! Great to hear that a fix has been put in place as of Flink 1.14. Since we are currently using Beam on top of Flink, we are currently limited to the Flink 1.13 runner, so I would expect the Fix not to be available to us yet. But to clarify the underlying proble

Re: Checkpoints aborted - Job is not in state RUNNING but FINISHED

2021-11-25 Thread Yun Gao
Hi Jonas, Previously Flink indeed does not support checkpoints after some tasks finished. In 1.14 we implement a first version for this feature (namely https://issues.apache.org/jira/browse/FLINK-2491), and it could be enabled by set execution.checkpointing.checkpoints-after-tasks-finish.enable