cc private@h.a.o. On Wed, Apr 24, 2024 at 3:35 PM ZanderXu <zande...@apache.org> wrote: > > Here are some summaries about the first phase: > 1. There are no big changes in this phase > 2. This phase just uses FS lock and BM lock to replace the original global > lock > 3. It's useful to improve the performance, since some operations just need > to hold FS lock or BM lock instead of the global lock > 4. This feature is turned off by default, you can enable it by setting > dfs.namenode.lock.model.provider.class to > org.apache.hadoop.hdfs.server.namenode.fgl.FineGrainedFSNamesystemLock > 5. This phase is very import for the ongoing development of the entire FGL > > Here I would like to express my special thanks to @kokonguyen191 and > @yuanboliu for their contributions. And you are also welcome to join us > and complete it together. > > > On Wed, 24 Apr 2024 at 14:54, ZanderXu <zande...@apache.org> wrote: > > > Hi everyone > > > > All subtasks of the first phase of the FGL have been completed and I plan > > to merge them into the trunk and start the second phase based on the trunk. > > > > Here is the PR that used to merge the first phases into trunk: > > https://github.com/apache/hadoop/pull/6762 > > Here is the ticket: https://issues.apache.org/jira/browse/HDFS-17384 > > > > I hope you can help to review this PR when you are available and give some > > ideas. > > > > > > HDFS-17385 <https://issues.apache.org/jira/browse/HDFS-17385> is used for > > the second phase and I have created some subtasks to describe solutions for > > some problems, such as: snapshot, getListing, quota. > > You are welcome to join us to complete it together. > > > > > > ---------- Forwarded message --------- > > From: Zengqiang XU <zande...@apache.org> > > Date: Fri, 2 Feb 2024 at 11:07 > > Subject: Discussion about NameNode Fine-grained locking > > To: <hdfs-dev@hadoop.apache.org> > > Cc: Zengqiang XU <xuzengqiang5...@gmail.com> > > > > > > Hi everyone > > > > I have started a discussion about NameNode Fine-grained Locking to improve > > performance of write operations in NameNode. > > > > I started this discussion again for serval main reasons: > > 1. We have implemented it and gained nearly 7x performance improvement in > > our prod environment > > 2. Many other companies made similar improvements based on their internal > > branch. > > 3. This topic has been discussed for a long time, but still without any > > results. > > > > I hope we can push this important improvement in the community so that all > > end-users can enjoy this significant improvement. > > > > I'd really appreciate you can join in and work with me to push this > > feature forward. > > > > Thanks very much. > > > > Ticket: HDFS-17366 <https://issues.apache.org/jira/browse/HDFS-17366> > > Design: NameNode Fine-grained locking based on directory tree > > <https://docs.google.com/document/d/1X499gHxT0WSU1fj8uo4RuF3GqKxWkWXznXx4tspTBLY/edit?usp=sharing> > >
--------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
