Hi Rinat, Savepoint need to be triggered when you want to create point in time which you want to use in future to revert back your state, also you could cancel job with savepoint which makes sure that you won’t lose any data during canceling job.
About your question that flink delete metadata file: I am using Flink 1.8.0 and Flink deletes metadata file when I cancel job with savepoint, It seems Flink doesn’t see any reason to keep checkpoint cause you created savepoint. When you just cancel job without savepoint flink shouldn’t delete metadata file from checkpoint. From: Rinat <r.shari...@cleverdata.ru> Sent: Wednesday, June 5, 2019 4:36 PM To: user <user@flink.apache.org> Subject: [Flink 1.6.1] _metadata file in retained checkpoint Hi mates, I’m trying to configure my job to retain checkpoints on it’s cancellation and got some troubles. I got the following args why not to use savepoints: 1. we already have all the job state on DFS in checkpoints directory 2. I can multiply size of the state on 2, when stopping the job, because the state is already stored in checkpoints dir, and I’ll save it one more time into savepoint dir 3. creation of checkpoints is incremental and savepoints - is not, so in my case (10 Tb state) the process of savepoint creation will took too long time 4. As I know, we can rescale or job from retained checkpoints I've configured my job to retain checkpoints on job cancelation and found an interesting issue - _metadata file is removed, when job is cancelled. So, I couldn’t restore my job from the retained checkpoint, is it an expected behaviour ? If so, what is wrong ? Sincerely yours, Rinat Sharipov Software Engineer at 1DMP CORE Team email: r.shari...@cleverdata.ru<mailto:a.totma...@cleverdata.ru> mobile: +7 (925) 416-37-26 CleverDATA make your data clever
