[
https://issues.apache.org/jira/browse/FLINK-13856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16915678#comment-16915678
]
Yun Tang commented on FLINK-13856:
----------------------------------
[~andrew_lin] For savepoint, as savepoint is not allowed to discard on subume,
there is no benefits for this improvement.
For checkpoint, we cannot just delete the checkpoint folder recursively by one
call for RocksDB incremental snapshot. If the checkpoint is RocksDB full
snapshot, we could delete the checkpoint folder by calling
{{storageLocation.disposeStorageLocation()}}. However, we cannot judge whether
the checkpoint is full now from current {{CheckpointProperties}}. Even though
we could introduce new field in {{CheckpointProperties}} to describe whether is
full checkpoint, we still have a concept to face which described in
{{CompletedCheckpointStorageLocation}} interface:
{code:java}
/**
* Disposes the storage location. This method should be called after all state
objects have
* been released. It typically disposes the base structure of the checkpoint
storage,
* like the checkpoint directory.
*/
void disposeStorageLocation() throws IOException;
{code}
Is it expected to directly dispose the base storage location without releasing
all state objects?
Since [~StephanEwen] introduced the definition for this method, I think he
might provide more insights.
> Reduce the delete file api when the checkpoint is completed
> -----------------------------------------------------------
>
> Key: FLINK-13856
> URL: https://issues.apache.org/jira/browse/FLINK-13856
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
> Affects Versions: 1.8.1, 1.9.0
> Reporter: andrew.D.lin
> Priority: Major
> Attachments: f6cc56b7-2c74-4f4b-bb6a-476d28a22096.png
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> When the new checkpoint is completed, an old checkpoint will be deleted by
> calling CompletedCheckpoint.discardOnSubsume().
> When deleting old checkpoints, follow these steps:
> 1, drop the metadata
> 2, discard private state objects
> 3, discard location as a whole
> In some cases, is it possible to delete the checkpoint folder recursively by
> one call?
> As far as I know the full amount of checkpoint, it should be possible to
> delete the folder directly.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
