Andrey Yarovoy created HDDS-16220:
-------------------------------------
Summary: bound per-transaction bucket-write-lock hold in
DirectoryDeletingService/KeyDeletingService
Key: HDDS-16220
URL: https://issues.apache.org/jira/browse/HDDS-16220
Project: Apache Ozone
Issue Type: Improvement
Components: Ozone Manager
Reporter: Andrey Yarovoy
{{DirectoryDeletingService}} submits an entire deletion run as one
{{PurgeDirectoriesRequest}} ({{{}DirectoryDeletingService#submitPurgePaths{}}}
→ {{{}addAllDeletedPath{}}}), and the apply handler
{{OMDirectoriesPurgeRequestWithFSO#validateAndUpdateCache}} acquires write
locks on all buckets in the batch up front and holds them across the entire
path × sub-dir × sub-file loop. The batch is bounded only by a byte budget
({{{}ratisByteLimit = 0.9 × ozone.om.ratis.log.appender.queue.byte-limit{}}},
default 32 MB → ~28.8 MB), and a single densely-populated directory can fill
that budget by itself.
The lock hold on the apply thread is proportional to the size of a single purge
transaction, and that size is effectively unbounded for a directory with a very
large number of direct children. There is no cap on the number of entries
applied under one lock acquisition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
