+1 thanks for the great work Rakesh and team. Regards, Uma
On Mon, Apr 5, 2021 at 2:44 AM Rakesh Radhakrishnan <rake...@apache.org> wrote: > Hi All, > > I would like to propose prefix based FileSystem Optimizations (FSO) work > HDDS-2939 <https://issues.apache.org/jira/browse/HDDS-2939> merging into > Ozone master branch. This optimization is intended to support atomic rename > and delete operations in Ozone namespace. > > Presently, a rename/delete operation can become prohibitively expensive for > such directories which have large sub-trees/sub-paths. This optimization > allows to perform rename, delete of any directory in a > deterministic/constant time atomically. > > The main functionality has been implemented and supports o3fs, ofs and > ObjectStore APIs, for details please refer to sub-tasks of HDDS-2939 which > we have been actively working in the feature branch > <https://github.com/apache/ozone/tree/HDDS-2939> for the last several > months. There is a configuration to turn this feature ON and OFF while > merging back to master, it will be turned OFF by default for now. > > Merging back to master will greatly help in stabilizing/testing the feature > more rigorously while ensuring the ongoing work on the master branch is not > negatively impacted by the feature as it is turned OFF by default. > > Following graph compares the performance between master(V0) and new > optimized code(V1) for the delete op in an unsecure cluster. Rename op also > has similar results. Please refer to the attached test report > < > https://issues.apache.org/jira/secure/attachment/13023395/Performance+Comparison+Between++Master+and+HDDS-2939+branch-Report-001.pdf > >in > jira for more details. > > *Ozone wiki page:* > > https://cwiki.apache.org/confluence/display/OZONE/FileSystem+Optimizations+-+HDDS-2939 > > Number of JIRAs Resolved: 27 > > Patch Available/WIP: 3 jira > > There are few ongoing sub-tasks, IMHO they are blockers for merge. We will > be actively working on these sub-tasks. > > Detailed design document is available in JIRA : > > > https://issues.apache.org/jira/secure/attachment/12991926/Ozone%20FS%20Namespace%20Proposal%20v1.0.docx > > > https://issues.apache.org/jira/secure/attachment/13023399/OzoneFS%20Optimizations_DesignOverview_%20HDDS-2939.pdf > > Quick stats on combined HDDS-2939 Patch : Commits > > 8b8a7e3ec84ad14e7128ff5fb3ded542ea9080d7..6d76676d2956346895de630a96c182188f9f3e38 > > 157 files changed, 16289 insertions(+), 776 deletions(-) > > Added/modified test cases= ~160 > > Thanks a lot to all contributors/reviewers. > > So, I feel this branch is ready to merge into master branch. Could you > please provide your feedback. If there are no objections, I will proceed > for merge voting. > > Thanks, > > Rakesh >
