[
https://issues.apache.org/jira/browse/HDDS-16220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-16220:
----------------------------------
Labels: pull-request-available (was: )
> bound per-transaction bucket-write-lock hold in
> DirectoryDeletingService/KeyDeletingService
> --------------------------------------------------------------------------------------------
>
> Key: HDDS-16220
> URL: https://issues.apache.org/jira/browse/HDDS-16220
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Manager
> Reporter: Andrey Yarovoy
> Assignee: Andrey Yarovoy
> Priority: Major
> Labels: pull-request-available
>
> {{DirectoryDeletingService}} submits an entire deletion run as one
> {{PurgeDirectoriesRequest}}
> ({{{}DirectoryDeletingService#submitPurgePaths{}}} →
> {{{}addAllDeletedPath{}}}), and the apply handler
> {{OMDirectoriesPurgeRequestWithFSO#validateAndUpdateCache}} acquires write
> locks on all buckets in the batch up front and holds them across the entire
> path × sub-dir × sub-file loop. The batch is bounded only by a byte budget
> ({{{}ratisByteLimit = 0.9 × ozone.om.ratis.log.appender.queue.byte-limit{}}},
> default 32 MB → ~28.8 MB), and a single densely-populated directory can fill
> that budget by itself.
>
> The lock hold on the apply thread is proportional to the size of a single
> purge transaction, and that size is effectively unbounded for a directory
> with a very large number of direct children. There is no cap on the number of
> entries applied under one lock acquisition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
