+1 for merge. Thanks, Bharat
On Wed, May 25, 2022 at 1:41 PM Uma Maheswara Rao Gangumalla < umaganguma...@gmail.com> wrote: > +1 > > Regards, > Uma > > On Wed, May 25, 2022 at 2:15 AM Sammi Chen <sammic...@apache.org> wrote: > > > Dear Ozone Devs, > > > > I 'm starting this thread to vote for the merge of Ozone Datanode RocksDB > > merge feature branch (HDDS-3630 > > https://issues.apache.org/jira/browse/HDDS-3630) into the master branch. > > > > Currently there will be one RocksDB for each Container on datanode, which > > leads to hundreds of thousands of RocksDB instances on one datanode. It's > > very challenging to manage this amount of RocksDB instances in one JVM. > > Please refer to the "problem statement" section of the design document[1] > > for challenge details. Unlike the current approach, Datanode RocksDB > merge > > feature will use only one RocksDB for each data volume. With far fewer > > RocksDB instances to manage, the write path performance and DN stability > > are improved, Refer to the Micro Benchmark Data section of the design > > document[1]. > > > > The feature has been developed in the last 8 months. And it has also been > > deployed in Tencent production for 5 months. Now we have reviewed all > core > > patches, and merged them into the feature branch, including new disk > > layout, new container schema, container replication, container deletion, > > block deletion, container scanner, non-rolling upgrade. There are no > > blocking JIRAs left. Document and the acceptance test are added too. > > > > For the sake of master branch feature stability, we provide a > configuration > > to enable/disable this feature. Currently by default, this feature is > > disabled. We can turn it on later once we believe it is stable enough. > > > > For more information, please check out the Merge RocksDB in datanode > > feature wiki page here: > > > > > > > https://cwiki.apache.org/confluence/display/OZONE/Merge+Rocksdb+in+Datanode+%28HDDS-3630%29+Merge+Checklist > > > > Great thanks to Mark Gui<guim...@126.com>, majority feature > investigation, > > design and core development is contributed by him. Thanks a lot to Runzhi > > Wang, for first finding current schema V2 layout challenges and > initiating > > the proposal of merge RocksDB, to Nanda, Mukul, Wei-Chiu, Jackson and > > Stephen O’Donnell for the help in design review, design discussion and > code > > review. If I miss anyone here, sorry for that, but your efforts are > well > > appreciated. > > > > > > [1] Merge RocksDB in Datanode Design Doc > > < > > > https://docs.google.com/document/d/1oPCRUVqTJDbYITbwvoVCOZ2-k108FwOXVhq9ukDKHbQ/edit#heading=h.w06vnlv65sb6 > > > > > > > > > Thanks, > > Sammi > > >
