Re: Checkpointing on cluster shutdown

2018-06-05 Thread Chesnay Schepler
If a TM goes down any data generated after the last successful checkpoint cannot be guaranteed to be consistent across the cluster. Hence, this data is discarded and we go back to the last known consistent state, the last checkpoint that was successfully created. On 05.06.2018 13:06, Garvit Sha

Re: Checkpointing on cluster shutdown

2018-06-05 Thread Garvit Sharma
But job should be terminated gracefully. Why is this behavior not there? On Tue, Jun 5, 2018 at 4:19 PM, Chesnay Schepler wrote: > No checkpoint will be triggered when the cluster is shutdown. For this > case you will have to manually trigger a savepoint. > > If a TM goes down it does not create

Re: Checkpointing on cluster shutdown

2018-06-05 Thread Chesnay Schepler
No checkpoint will be triggered when the cluster is shutdown. For this case you will have to manually trigger a savepoint. If a TM goes down it does not create a checkpoint. IN these cases the job will be restarted from the last successful checkpoint. On 05.06.2018 12:01, Data Engineer wrote:

Checkpointing on cluster shutdown

2018-06-05 Thread Data Engineer
Hi, Suppose I have a working Flink cluster with 1 taskmanager and 1 jobmanager and I have enabled checkpointing with say an interval of 1 minute. Now if I shut down the Flink cluster in between checkpoints (say for some upgrade), will the JobManager automatically trigger a checkpoint before going