Hi Arvid,
The flink document says,
"ExternalizedCheckpointCleanup.DELETE_ON_CANCELLATION: Delete the
checkpoint when the job is cancelled. The checkpoint state will only be
available if the job fails."
If I have to resume the program after cancellation, Do I have to use
ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION or I have to use a
save point ? If I have to use a save point, then what is the use of
ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION ?
Thanks,
Sudhansu
On Tue, May 11, 2021 at 7:47 PM Arvid Heise <ar...@apache.org> wrote:
> Hi Sudhansu,
>
> if you don't set RETAIN_ON_CANCELLATION, the folder should be cleaned up
> automatically. If you explicitly want to retain the checkpoint, then there
> is not much that Flink can do or I may have misunderstood you.
>
> On Tue, May 11, 2021 at 4:09 PM sudhansu jena <sudhansu.jena...@gmail.com>
> wrote:
>
>> Hi Team,
>>
>> We have recently enabled Check Pointing in our flink job using
>> FSStateBackend pointing to S3 bucket.
>>
>> Below is the sample code for enabling the checkpointing for the job. The
>> query is each time we cancel the job and restart from the flink dashboard,
>> a new folder is getting created along with the old checkpointing folder in
>> the S3 bucket, So is there a way to get rid of these old checkpointed
>> folders automatically assuming they are not gonna be used for restoring the
>> state except the latest folder?
>>
>> env.setStateBackend(new
>> FsStateBackend("s3://flinkcheckpointing/job-name/",true));
>> env.enableCheckpointing(1000);
>> Class<?> unmodColl =
>> Class.forName("java.util.Collections$UnmodifiableCollection");
>> env.getConfig().addDefaultKryoSerializer(unmodColl,
>> UnmodifiableCollectionsSerializer.class);
>> CheckpointConfig config = env.getCheckpointConfig();
>>
>> config.enableExternalizedCheckpoints(CheckpointConfig.ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION);
>> config.setCheckpointingMode(CheckpointingMode.AT_LEAST_ONCE);
>>
>>
>> Thanks,
>> Sudhansu
>>
>
