[
https://issues.apache.org/jira/browse/HDFS-11922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang resolved HDFS-11922.
--------------------------------
Resolution: Done
Fix Version/s: HDFS-7240
Target Version/s: HDFS-7240
> Ozone: KSM: Garbage collect deleted blocks
> ------------------------------------------
>
> Key: HDFS-11922
> URL: https://issues.apache.org/jira/browse/HDFS-11922
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Reporter: Anu Engineer
> Assignee: Weiwei Yang
> Priority: Critical
> Fix For: HDFS-7240
>
> Attachments: Asynchronous key delete .pdf
>
>
> We need to garbage collect deleted blocks from the Datanodes. There are two
> cases where we will have orphaned blocks. One is like the classical HDFS,
> where someone deletes a key and we need to delete the corresponding blocks.
> Another case, is when someone overwrites a key -- an overwrite can be treated
> as a delete and a new put -- that means that older blocks need to be GC-ed at
> some point of time.
> Couple of JIRAs has discussed this in one form or another -- so consolidating
> all those discussions in this JIRA.
> HDFS-11796 -- needs to fix this issue for some tests to pass
> HDFS-11780 -- changed the old overwriting behavior to not supporting this
> feature for time being.
> HDFS-11920 - Once again runs into this issue when user tries to put an
> existing key.
> HDFS-11781 - delete key API in KSM only deletes the metadata -- and relies on
> GC for Datanodes.
> When we solve this issue, we should also consider 2 more aspects.
> One, we support versioning in the buckets, tracking which blocks are really
> orphaned is something that KSM will do. So delete and overwrite at some point
> needs to decide how to handle versioning of buckets.
> Two, If a key exists in a closed container, then it is immutable, hence the
> strategy of removing the key might be more complex than just talking to an
> open container.
> cc : [~xyao], [~cheersyang], [~vagarychen], [~msingh], [~yuanbo],
> [~szetszwo], [~nandakumar131]
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
