A0%E6%B5%A9%E5%8D%9A&uid=hfutzhanghb%40163.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fsmc804eb39b0e7885aa8801c3bb66e497d.jpg&items=%5B%22hfutzhanghb%40163.com%22%5D>
>>
>> Replied Message
>> From haiyang hu
>> Date 12/31/2024 23:08
>> To Ay
> From haiyang hu
> Date 12/31/2024 23:08
> To Ayush Saxena
> Cc Hui Fei ,
> ZanderXu ,
> Hdfs-dev ,
> ,
> Xiaoqiao He ,
> slfan1989 ,
>
> Subject Re: Discussion about NameNode Fine-grained locking
> Thanks for your hard work and push it forward
:08 |
| To | Ayush Saxena |
| Cc | Hui Fei ,
ZanderXu ,
Hdfs-dev ,
,
Xiaoqiao He ,
slfan1989 ,
|
| Subject | Re: Discussion about NameNode Fine-grained locking |
Thanks for your hard work and push it forward.
It looks good, +1 for merging phase 1 codes, hope we can work together to
promote this
;>>> >>
> >>>>>> >> Best Regards,
> >>>>>> >> - He Xiaoqiao
> >>>>>> >>
> >>>>>> >> On Fri, Apr 26, 2024 at 3:50 PM Hui Fei
> wrote:
> >>>>>> >> >
> >>>>>
gt;> >> > Thanks for interest and advice on this.
>>>>>> >> >
>>>>>> >> > Just would like to share some info here
>>>>>> >> >
>>>>>> >> > ZanderXu leads this feature and
Yuanboliu and Kokonguyen191 are also very
>>>>> >> familiar with this feature. We discussed many details offline.
>>>>> >> >
>>>>> >> > Welcome to more people interested in joining the development and
>>>
lready
>>>> >> >> been practiced in their prod environment. We have also practiced
>>>> it in
>>>> >> our
>>>> >> >> prod environment and gained benefits, and we are also willing to
>>>> spend
>>>> >> a
&g
as long as the
>>> >> overall
>>> >> >> solution is right.
>>> >> >> 6. I can spend a lot of time leading the promotion of this entire
>>> >> feature
>>> >> >> and I hope more people can join us
> >> > However, I still have some concerns, which roughly involve the
>> >> following
>> >> >> > aspects:
>> >> >> >
>> >> >> > 1. While ZanderXu and Hui Fei have deep expertise in HDFS and are
>> >&
d to everything going smoothly in the future.
> >> >> >
> >> >> > Best Regards,
> >> >> > - Shilun Fan.
> >> >> >
> >> >> > On Wed, Apr 24, 2024 at 3:51 PM Xiaoqiao He >
> >> wrote:
> >> >> >
>
; > lock
>> >> >> > 3. It's useful to improve the performance, since some operations
>> just
>> >> >> need
>> >> >> > to hold FS lock or BM lock instead of the global lock
>> >> >> > 4. This feature is turned off by default, you c
the
> entire
> >> >> FGL
> >> >> >
> >> >> > Here I would like to express my special thanks to @kokonguyen191
> and
> >> >> > @yuanboliu for their contributions. And you are also welcome to
> join us
> >> >&
nderXu wrote:
>> >> >
>> >> > > Hi everyone
>> >> > >
>> >> > > All subtasks of the first phase of the FGL have been completed and I
>> >> plan
>> >> > > to merge them into the trunk and start the seco
; >> > > Here is the PR that used to merge the first phases into trunk:
> >> > > https://github.com/apache/hadoop/pull/6762
> >> > > Here is the ticket:
> https://issues.apache.org/jira/browse/HDFS-17384
> >> > >
> >> > > I hope you can help
based on the
>>> trunk.
>>> > >
>>> > > Here is the PR that used to merge the first phases into trunk:
>>> > > https://github.com/apache/hadoop/pull/6762
>>> > > Here is the ticket: https://issues.apache.org/jira/browse/HD
pache.org/jira/browse/HDFS-17384
>> > >
>> > > I hope you can help to review this PR when you are available and give
>> some
>> > > ideas.
>> > >
>> > >
>> > > HDFS-17385 <https://issues.apache.org/jira/browse/HDFS-17385&
; solutions for
> > > some problems, such as: snapshot, getListing, quota.
> > > You are welcome to join us to complete it together.
> > >
> > >
> > > -- Forwarded message -
> > > From: Zengqiang XU
> > > Date: Fri
; > HDFS-17385 <https://issues.apache.org/jira/browse/HDFS-17385> is used for
> > the second phase and I have created some subtasks to describe solutions for
> > some problems, such as: snapshot, getListing, quota.
> > You are welcome to join us to complete it together.
> &
quota.
> You are welcome to join us to complete it together.
>
>
> -- Forwarded message -
> From: Zengqiang XU
> Date: Fri, 2 Feb 2024 at 11:07
> Subject: Discussion about NameNode Fine-grained locking
> To:
> Cc: Zengqiang XU
>
>
> Hi everyone
>
&
uch as: snapshot, getListing, quota.
You are welcome to join us to complete it together.
-- Forwarded message -
From: Zengqiang XU
Date: Fri, 2 Feb 2024 at 11:07
Subject: Discussion about NameNode Fine-grained locking
To:
Cc: Zengqiang XU
Hi everyone
I have started a discussion
> > 3. Each milestone is developed based on the latest trunk
> branch
> > to
> > > > >> reduce
> > > > >> > > conflicts
> > > > >> > >
> > > > >> > > If you have any concerns, please feel free to discuss them
> >
rns, please feel free to discuss them
> > together.
> > > >> > > I hope you can join us to push this project forward together,
> > > thanks.
> > > >> > >
> > > >> > >
> > > >> > > On Mon, 5 Feb 2024 at 11:33,
you for raising the issue of this long-standing
> bottleneck,
> > >> this
> > >> > > > will be a very important improvement!
> > >> > > >
> > >> > > > Hopefully can participate and push forward together.
> > >> > >
; >> > > > Brahma Reddy Battula 于2024年2月3日周六 00:40写道：
> >> > > >
> >> > > >> Thanks for bringing this and considering all the history around
> >> this.
> >> > > >> One of the outstanding bottleneck(global lock) fro
we can push forward this time.
>> > > >>
>> > > >>
>> > > >> On Fri, Feb 2, 2024 at 12:23 PM Hui Fei
>> > wrote:
>> > > >>
>> > > >> > Thanks for driving this. It's very meaningful. The p
> > >> improvement
> > > >> > looks very good.
> > > >> >
> > > >> > Many users are facing the write performance issue. As far as I
> know,
> > > >> some
> > > >> > companies already implemented the similar id
ready implemented the similar idea on their internal
> > >> branches.
> > >> > But the internal branch is very different from the community one. So
> > >> it's
> > >> > very hard to be in sync with the community. If this improve
from the community one. So
> >> it's
> >> > very hard to be in sync with the community. If this improvement can be
> >> > involved in the community, that would be great to both end-user and
> the
> >> > community.
> >> >
> >&
internal
>> branches.
>> > But the internal branch is very different from the community one. So
>> it's
>> > very hard to be in sync with the community. If this improvement can be
>> > involved in the community, that would be great to both end-user and
hard to be in sync with the community. If this improvement can be
> > involved in the community, that would be great to both end-user and the
> > community.
> >
> > It is very worth doing.
> >
> > Zengqiang XU 于2024年2月2日周五 11:07写道：
> >
> > &
ery hard to be in sync with the community. If this improvement can be
> involved in the community, that would be great to both end-user and the
> community.
>
> It is very worth doing.
>
> Zengqiang XU 于2024年2月2日周五 11:07写道：
>
> > Hi everyone
> >
> > I have
m the community one. So it's
very hard to be in sync with the community. If this improvement can be
involved in the community, that would be great to both end-user and the
community.
It is very worth doing.
Zengqiang XU 于2024年2月2日周五 11:07写道：
> Hi everyone
>
> I have started a discussion
Hi everyone
I have started a discussion about NameNode Fine-grained Locking to improve
performance of write operations in NameNode.
I started this discussion again for serval main reasons:
1. We have implemented it and gained nearly 7x performance improvement in
our prod environment
2. Many
33 matches
Mail list logo
