Discusstion and Review for reverting HDFS-9260 FoldedTreeSet

2021-06-03 Thread Hui Fei
Hi Community, HDFS-9260's performance issue has existed for years, and many companies using hadoop 3.x encounter it. Maybe it's time to revert FoldedTreeSet. Now the PR from Haibin Huang is ready, please see https://github.com/apache/hadoop/pull/3065 Does it make sense? it is a huge

Re: [DISCUSS] Hadoop 3.3.2 release?

2021-10-28 Thread Hui Fei
Thank you, Chao! Do you need any help? I can assist you. Chao Sun 于2021年9月8日周三 上午1:06写道: > Hi all, > > It has been almost 3 months since the 3.3.1 release and branch-3.3 has > accumulated quite a few commits (118 atm). In particular, Spark community > recently found an issue which prevents one f

Re: [ANNOUNCE] Apache Hadoop 3.3.2 release

2022-03-03 Thread Hui Fei
Great work! Thanks, Chao! Chao Sun 于2022年3月4日周五 09:30写道: > Hi All, > > It gives me great pleasure to announce that the Apache Hadoop community has > voted to release Apache Hadoop 3.3.2. > > This is the second stable release of Apache Hadoop 3.3 line. It contains > 284 bug fixes, improvements an

Re: [Announcement] DiDi HDFS successfully upgraded from 2.7.2 to 3.2.0

2019-12-09 Thread Hui Fei
Hi Eric, It's Rolling Upgrade without downtime. We have a cluster with thousands of nodes and 5 namespace, it takes 1 month to do the upgrade. Upgrading Service includes JournalNode, NameNode, ZKFC and DataNode. Related issues of rolling upgrade are HDFS-13596,HDFS-14396,HDFS-14831,HDFS-14509. Wa

Re: Client Caller Context Through Router(RBF)

2020-05-07 Thread Hui Fei
Ayush, Thanks for bringing this up, it is very meaningful! Add one more typical problem, NameNode should log both real client and Router 's ip. But now NN just logs router's ip, it's difficult for troubleshooting Ayush Saxena 于2020年5月4日周一 下午5:26写道: > Hi All, > Wanted to share and discuss a pr

Re: [DISCUSS] fate of branch-2.9

2020-08-28 Thread Hui Fei
+1 It's almost three years since Hadoop 3.0.0 was released. Hadoop 3.x is stable enough and many companies have upgraded their cluster to 3.x It's time to encourage people to upgrade to Hadoop 3.x Wei-Chiu Chuang 于2020年3月3日周二 上午9:12写道: > Hi, > > Following the discussion to end branch-2.8, I want

Re: [DISCUSS] Ozone TLP proposal

2020-09-23 Thread Hui Fei
Hi Elek, > 2. Following the path of Submarine, any existing Hadoop committers -- who are willing to contribute -- can ask to be included in the initial committer list without any additional constraints. (Edit the wiki, or send an email to this thread or to me). Thanks for Vinod to suggesting this

Re: [VOTE] Release Apache Hadoop 3.2.2 - RC4

2020-12-18 Thread Hui Fei
+1 (non-binding) - Build from source - checksums and signatures - Setup a pseudo cluster and run shell test Thanks Xiaoqiao He 于2020年12月10日周四 上午1:01写道: > Hi folks, > > The release candidate (RC4) for Hadoop-3.2.2 is available now. > There are 10 commits[1] differences between RC4 and RC3[2]. >

Re: [VOTE] Release Apache Hadoop 3.2.2 - RC5

2021-01-06 Thread Hui Fei
+1 (non-binding) - Build from source - Verify checksums and signatures - Setup a pseudo cluster and run shell test Thanks Xiaoqiao He 于2021年1月3日周日 下午10:40写道: > Hi folks, > > The release candidate (RC5) for Hadoop-3.2.2 is available now. > There are 4 commits[1] differences between RC5 and RC4[

Can't find the issue id of Fix potential FSImage corruption. Contributed by Daryn Sharp.

2019-04-21 Thread Hui Fei
HI All I saw one commit as follow commit 8a41edb089fbdedc5e7d9a2aeec63d126afea49f Author: Vinayakumar B Date: Mon Oct 15 15:48:26 2018 +0530 Fix potential FSImage corruption. Contributed by Daryn Sharp. But i can't find the matching issue. Could anyone tell me ? Thanks!

Re: Can't find the issue id of Fix potential FSImage corruption. Contributed by Daryn Sharp.

2019-04-21 Thread Hui Fei
Another commit commit 0a1e922f3d8eca4e852be57124ec1bcafaadb289 Author: Konstantin V Shvachko Date: Mon Jul 16 18:20:24 2018 -0700 Fix potential FSImage corruption. Contributed by Ekanth Sethuramalingam & Arpit Agarwal. Hui Fei 于2019年4月22日周一 上午11:25写道: > HI All > I saw one

Re: [DISCUSS] Hadoop 2019 Release Planning

2019-08-16 Thread Hui Fei
Hi Wangda, Thanks for bringing this up! Looking forward to see HDFS 3.x is widely used,but RollingUpgrade is a problem. Hope commiters watch and review these issues, Thanks https://issues.apache.org/jira/browse/HDFS-13596 https://issues.apache.org/jira/browse/HDFS-14396 Wangda Tan 于2019年8月10日周六 上

Re: [VOTE] Hadoop 3.2.x EOL

2023-12-06 Thread Hui Fei
+1 Xiaoqiao He 于2023年12月6日周三 12:10写道: > Dear Hadoop devs, > > Given the feedback from the discussion thread [1], I'd like to start > an official thread for the community to vote on release line 3.2 EOL. > > It will include, > a. An official announcement informs no further regular Hadoop 3.2.x >

Re: Discussion about NameNode Fine-grained locking

2024-02-01 Thread Hui Fei
Thanks for driving this. It's very meaningful. The performance improvement looks very good. Many users are facing the write performance issue. As far as I know, some companies already implemented the similar idea on their internal branches. But the internal branch is very different from the commun

Re: Discussion about NameNode Fine-grained locking

2024-03-05 Thread Hui Fei
ou for raising the issue of this long-standing bottleneck, > this > > > > will be a very important improvement! > > > > > > > > Hopefully can participate and push forward together. > > > > > > > > Best Regards~ > > > > > > > > Brahm

Re: Discussion about NameNode Fine-grained locking

2024-04-26 Thread Hui Fei
; > > Thank you for your hard work! This is a very meaningful improvement, and > > from the design document, we can see a significant increase in HDFS > > read/write throughput. > > > > I am happy to see the progress made on HDFS-17384. > > > > However, I

Re: Discussion about NameNode Fine-grained locking

2024-05-06 Thread Hui Fei
Pool, we can't let it > grow > > infinitely in case of heavy loads and we need to have some auto > > throttling mechanism for it > > * I didn't catch your Storage Policy problem. If I decode it right, the > > problem is like the policy could be set on an ancestor

Re: Discussion about NameNode Fine-grained locking

2024-05-06 Thread Hui Fei
BTW, there is a Slack channel hdfs-fgl for this feature. can join it and discuss more details. Is it necessary to hold a meeting to discuss this? So that we can push it forward quickly. Agreed with ZanderXu, it seems inefficient to discuss details via email list. Hui Fei 于2024年5月6日周一 23:50写道

Re: Discussion about NameNode Fine-grained locking

2024-12-31 Thread Hui Fei
t; https://docs.google.com/document/d/1DXkiVxef9wCmICjpZyIQO-yxsgwc4wnf2lTKQ3UXe30/edit?tab=t.0#heading=h.zesd6omuu3kr > > Seeking Community Feedback > > Your questions and concerns are always welcome. > > We can discuss them in detail on the Slack Channel: > https://app.sla

Re: [DISCUSS] Request to merge branch HDFS-17531 into trunk.

2025-01-22 Thread Hui Fei
days. If it takes a long time, I think it may block the trunk release and All community members need to remember that there is documentation required. It doesn't look good. That's my thought, and we can wait for others' opinions jian zhang 于2025年1月22日周三 16:53写道: > Hi, Hu

Re: [DISCUSS] Request to merge branch HDFS-17531 into trunk.

2025-01-22 Thread Hui Fei
Thanks for your great work, looking forward to this feature. Some comments from me. - I checked and found that there are still 3 sub tasks under this feature jira ticket, are they necessary to be solved? - I didn't find the documentation for this feature. It's a key feature, Is it necessary to a

Re: [VOTE] Merge HDFS-17384 to trunk

2025-01-13 Thread Hui Fei
merge request: > >- *+1* : Approve the merge >- * -1* : Disapprove the merge (please provide a reason) >- * 0* : Abstain > > This vote will remain open for at least *1 week*, closing on *2025-01-17* > (or > until a consensus is reached). > > I

Re: [DISCUSS] Request to merge branch HDFS-17531 into trunk.

2025-02-13 Thread Hui Fei
> >> >> Hi, all, >> >> Currently this feature has been developed and passed the pipeline. >> Please >> >> continue to help review this feature. >> >> >> >> Best Regards, >> >> Jian Zhang >> >> >> >&g

Re: [VOTE] Request to merge branch HDFS-17531 into trunk.

2025-02-13 Thread Hui Fei
Did some test work referring to the documentation. - Compiled source codes, built a local cluster and the async feature worked fine. - it is disabled by default - can increase or decrease the thread number by changing the related configurations +1 jian zhang 于2025年2月14日周五 15:21写道

[jira] [Created] (HDFS-16060) There is an inconsistent between replicas of datanodes when hardware is abnormal

2021-06-09 Thread Hui Fei (Jira)
Hui Fei created HDFS-16060: -- Summary: There is an inconsistent between replicas of datanodes when hardware is abnormal Key: HDFS-16060 URL: https://issues.apache.org/jira/browse/HDFS-16060 Project: Hadoop

[jira] [Reopened] (HDFS-13671) Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet

2021-06-17 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei reopened HDFS-13671: > Namenode deletes large dir slowly caused by FoldedTreeSet#removeAnd

[jira] [Resolved] (HDFS-13671) Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet

2021-06-18 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-13671. Resolution: Fixed > Namenode deletes large dir slowly caused by FoldedTreeSet#removeAnd

[jira] [Created] (HDFS-16105) Edit log corruption due to mismatch between fileId and path

2021-06-30 Thread Hui Fei (Jira)
Hui Fei created HDFS-16105: -- Summary: Edit log corruption due to mismatch between fileId and path Key: HDFS-16105 URL: https://issues.apache.org/jira/browse/HDFS-16105 Project: Hadoop HDFS Issue

[jira] [Resolved] (HDFS-16105) Edit log corruption due to mismatch between fileId and path

2021-07-01 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16105. Resolution: Invalid > Edit log corruption due to mismatch between fileId and p

[jira] [Resolved] (HDFS-16106) Fix flaky unit test TestDFSShell

2021-07-02 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16106. Fix Version/s: 3.4.0 Resolution: Fixed > Fix flaky unit test TestDFSSh

[jira] [Resolved] (HDFS-16108) Incorrect log placeholders used in JournalNodeSyncer

2021-07-04 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16108. Fix Version/s: 3.4.0 Resolution: Fixed > Incorrect log placeholders used in JournalNodeSyn

[jira] [Resolved] (HDFS-16088) Standby NameNode process getLiveDatanodeStorageReport request to reduce Active load

2021-07-07 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16088. Fix Version/s: 3.4.0 Resolution: Fixed > Standby NameNode process getLiveDatanodeStorageRep

[jira] [Created] (HDFS-16166) TestDecommissionWithBackoffMonitor#testDecommissionWithCloseFileAndListOpenFiles fails

2021-08-12 Thread Hui Fei (Jira)
Hui Fei created HDFS-16166: -- Summary: TestDecommissionWithBackoffMonitor#testDecommissionWithCloseFileAndListOpenFiles fails Key: HDFS-16166 URL: https://issues.apache.org/jira/browse/HDFS-16166 Project

[jira] [Created] (HDFS-16167) TestDFSInotifyEventInputStreamKerberized#testWithKerberizedCluster fails

2021-08-12 Thread Hui Fei (Jira)
Hui Fei created HDFS-16167: -- Summary: TestDFSInotifyEventInputStreamKerberized#testWithKerberizedCluster fails Key: HDFS-16167 URL: https://issues.apache.org/jira/browse/HDFS-16167 Project: Hadoop HDFS

[jira] [Created] (HDFS-16168) TestHDFSFileSystemContract#testAppend fails

2021-08-12 Thread Hui Fei (Jira)
Hui Fei created HDFS-16168: -- Summary: TestHDFSFileSystemContract#testAppend fails Key: HDFS-16168 URL: https://issues.apache.org/jira/browse/HDFS-16168 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-16169) TestBlockTokenWithDFSStriped#testEnd2End fails

2021-08-12 Thread Hui Fei (Jira)
Hui Fei created HDFS-16169: -- Summary: TestBlockTokenWithDFSStriped#testEnd2End fails Key: HDFS-16169 URL: https://issues.apache.org/jira/browse/HDFS-16169 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-16170) TestFileTruncate#testTruncateWithDataNodesShutdownImmediately fails

2021-08-12 Thread Hui Fei (Jira)
Hui Fei created HDFS-16170: -- Summary: TestFileTruncate#testTruncateWithDataNodesShutdownImmediately fails Key: HDFS-16170 URL: https://issues.apache.org/jira/browse/HDFS-16170 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-16160) Improve the parameter annotation in DatanodeProtocol#sendHeartbeat

2021-08-18 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16160. Fix Version/s: 3.4.0 Resolution: Fixed > Improve the parameter annotation in DatanodeProto

[jira] [Resolved] (HDFS-15966) Empty the statistical parameters when emptying the redundant queue

2021-09-01 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-15966. Fix Version/s: 3.4.0 Resolution: Fixed > Empty the statistical parameters when emptying

[jira] [Resolved] (HDFS-16204) Improve FSDirEncryptionZoneOp related parameter comments

2021-09-06 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16204. Fix Version/s: 3.4.0 Resolution: Fixed > Improve FSDirEncryptionZoneOp related parameter comme

[jira] [Resolved] (HDFS-16210) RBF: Add the option of refreshCallQueue to RouterAdmin

2021-09-08 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16210. Fix Version/s: 3.4.0 Resolution: Fixed > RBF: Add the option of refreshCallQueue to RouterAd

[jira] [Reopened] (HDFS-16210) RBF: Add the option of refreshCallQueue to RouterAdmin

2021-09-08 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei reopened HDFS-16210: > RBF: Add the option of refreshCallQueue to RouterAd

[jira] [Resolved] (HDFS-16065) RBF: Add metrics to record Router's operations

2021-09-09 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16065. Fix Version/s: 3.4.0 Resolution: Fixed > RBF: Add metrics to record Router's op

[jira] [Resolved] (HDFS-16210) RBF: Add the option of refreshCallQueue to RouterAdmin

2021-09-09 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16210. Resolution: Fixed > RBF: Add the option of refreshCallQueue to RouterAd

[jira] [Reopened] (HDFS-16210) RBF: Add the option of refreshCallQueue to RouterAdmin

2021-09-10 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei reopened HDFS-16210: > RBF: Add the option of refreshCallQueue to RouterAd

[jira] [Resolved] (HDFS-16210) RBF: Add the option of refreshCallQueue to RouterAdmin

2021-09-11 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16210. Resolution: Fixed > RBF: Add the option of refreshCallQueue to RouterAd

[jira] [Resolved] (HDFS-16221) RBF: Add usage of refreshCallQueue for Router

2021-09-12 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16221. Fix Version/s: 3.4.0 Resolution: Fixed > RBF: Add usage of refreshCallQueue for Rou

[jira] [Resolved] (HDFS-16197) Simplify getting NNStorage in FSNamesystem

2021-09-12 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16197. Fix Version/s: 3.4.0 Resolution: Fixed > Simplify getting NNStorage in FSNamesys

[jira] [Reopened] (HDFS-16221) RBF: Add usage of refreshCallQueue for Router

2021-09-12 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei reopened HDFS-16221: > RBF: Add usage of refreshCallQueue for Rou

[jira] [Resolved] (HDFS-16221) RBF: Add usage of refreshCallQueue for Router

2021-09-12 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16221. Resolution: Fixed > RBF: Add usage of refreshCallQueue for Rou

[jira] [Resolved] (HDFS-16235) Deadlock in LeaseRenewer for static remove method

2021-09-25 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16235. Fix Version/s: 3.4.0 Resolution: Fixed > Deadlock in LeaseRenewer for static remove met

[jira] [Resolved] (HDFS-16236) Example command for daemonlog is not correct

2021-09-25 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16236. Fix Version/s: 3.4.0 Resolution: Fixed > Example command for daemonlog is not corr

[jira] [Resolved] (HDFS-16237) Record the BPServiceActor information that communicates with Standby

2021-09-27 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16237. Fix Version/s: 3.4.0 Resolution: Fixed > Record the BPServiceActor information that communica

[jira] [Resolved] (HDFS-16231) Fix TestDataNodeMetrics#testReceivePacketSlowMetrics

2021-09-27 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16231. Fix Version/s: 3.4.0 Resolution: Fixed > Fix TestDataNodeMetrics#testReceivePacketSlowMetr

[jira] [Resolved] (HDFS-16247) RBF: Fix the ProcessingAvgTime and ProxyAvgTime code comments and document metrics describe ms unit

2021-10-04 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16247. Fix Version/s: 3.4.0 Resolution: Fixed > RBF: Fix the ProcessingAvgTime and ProxyAvgTime c

[jira] [Resolved] (HDFS-16264) When adding block keys, the records come from the specific Block Pool

2021-10-11 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16264. Fix Version/s: 3.4.0 Resolution: Fixed > When adding block keys, the records come from the speci

[jira] [Resolved] (HDFS-16244) Add the necessary write lock in Checkpointer#doCheckpoint()

2021-10-13 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16244. Fix Version/s: 3.4.0 Resolution: Fixed > Add the necessary write lock in Checkpointer#doCheckpo

[jira] [Resolved] (HDFS-16281) Fix flaky unit tests failed due to timeout

2021-10-25 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16281. Fix Version/s: 3.4.0 Resolution: Fixed > Fix flaky unit tests failed due to time

[jira] [Resolved] (HDFS-16291) Make the comment of INode#ReclaimContext more standardized

2021-11-03 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16291. Fix Version/s: 3.4.0 Resolution: Fixed > Make the comment of INode#ReclaimContext more standardi

[jira] [Resolved] (HDFS-16296) RBF: RouterRpcFairnessPolicyController add denied permits for each nameservice

2021-11-04 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16296. Fix Version/s: 3.4.0 Resolution: Fixed > RBF: RouterRpcFairnessPolicyController add denied perm

[jira] [Resolved] (HDFS-16302) RBF: RouterRpcFairnessPolicyController record requests handled by each nameservice

2021-11-07 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16302. Fix Version/s: 3.4.0 Resolution: Fixed > RBF: RouterRpcFairnessPolicyController record reque

[jira] [Resolved] (HDFS-16305) Record the remote NameNode address when the rolling log is triggered

2021-11-11 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16305. Fix Version/s: 3.4.0 Resolution: Fixed > Record the remote NameNode address when the rolling log

[jira] [Resolved] (HDFS-16287) Support to make dfs.namenode.avoid.read.slow.datanode reconfigurable

2021-11-11 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16287. Fix Version/s: 3.4.0 Resolution: Fixed > Support to make dfs.namenode.avoid.read.slow.datan

[jira] [Resolved] (HDFS-16320) Datanode retrieve slownode information from NameNode

2021-11-22 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16320. Fix Version/s: 3.4.0 Resolution: Fixed > Datanode retrieve slownode information from NameN

[jira] [Resolved] (HDFS-16350) Datanode start time should be set after RPC server starts successfully

2021-11-24 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16350. Resolution: Fixed > Datanode start time should be set after RPC server starts successfu

[jira] [Reopened] (HDFS-16350) Datanode start time should be set after RPC server starts successfully

2021-11-24 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei reopened HDFS-16350: > Datanode start time should be set after RPC server starts successfu

[jira] [Resolved] (HDFS-16168) TestHDFSFileSystemContract#testAppend fails

2021-12-20 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16168. Fix Version/s: 3.4.0 Resolution: Fixed > TestHDFSFileSystemContract#testAppend fa

[jira] [Resolved] (HDFS-16392) TestWebHdfsFileSystemContract#testResponseCode fails

2021-12-21 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16392. Fix Version/s: 3.4.0 Resolution: Fixed > TestWebHdfsFileSystemContract#testResponseCode fa

[jira] [Resolved] (HDFS-16401) Remove the worthless DatasetVolumeChecker#numAsyncDatasetChecks

2022-01-25 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16401. Fix Version/s: 3.4.0 Resolution: Fixed > Remove the worthless DatasetVolumeChec

[jira] [Resolved] (HDFS-16444) Show start time of JournalNode on Web

2022-01-30 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16444. Fix Version/s: 3.4.0 Resolution: Fixed > Show start time of JournalNode on

[jira] [Resolved] (HDFS-16406) DataNode metric ReadsFromLocalClient does not count short-circuit reads

2022-02-08 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16406. Fix Version/s: 3.4.0 Resolution: Fixed > DataNode metric ReadsFromLocalClient does not count sh

[jira] [Resolved] (HDFS-16316) Improve DirectoryScanner: add regular file check related block

2022-02-21 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16316. Fix Version/s: 3.4.0 Resolution: Fixed > Improve DirectoryScanner: add regular file check rela

[jira] [Resolved] (HDFS-16480) Fix typo: indicies -> indices

2022-02-28 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16480. Fix Version/s: 3.4.0 Assignee: Jiale Qi Resolution: Fixed > Fix typo: indicies -> i

[jira] [Resolved] (HDFS-16458) [SPS]: Fix bug for unit test of reconfiguring SPS mode

2022-03-01 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16458. Fix Version/s: 3.4.0 Resolution: Fixed > [SPS]: Fix bug for unit test of reconfiguring SPS m

[jira] [Resolved] (HDFS-16499) [SPS]: Should not start indefinitely while another SPS process is running

2022-03-16 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16499. Fix Version/s: 3.4.0 Resolution: Fixed > [SPS]: Should not start indefinitely while another

[jira] [Resolved] (HDFS-16460) [SPS]: Handle failure retries for moving tasks

2022-04-07 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16460. Fix Version/s: 3.4.0 Resolution: Fixed > [SPS]: Handle failure retries for moving ta

[jira] [Resolved] (HDFS-16488) [SPS]: Expose metrics to JMX for external SPS

2022-04-24 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16488. Fix Version/s: 3.4.0 Resolution: Fixed > [SPS]: Expose metrics to JMX for external

[jira] [Reopened] (HDFS-16488) [SPS]: Expose metrics to JMX for external SPS

2022-04-24 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei reopened HDFS-16488: > [SPS]: Expose metrics to JMX for external

[jira] [Resolved] (HDFS-16488) [SPS]: Expose metrics to JMX for external SPS

2022-04-25 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16488. Resolution: Fixed > [SPS]: Expose metrics to JMX for external

[jira] [Resolved] (HDFS-16525) System.err should be used when error occurs in multiple methods in DFSAdmin class

2022-05-12 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16525. Fix Version/s: 3.4.0 Resolution: Fixed > System.err should be used when error occurs in multi

[jira] [Resolved] (HDFS-16671) RBF: RouterRpcFairnessPolicyController supports configurable permit acquire timeout

2022-07-28 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16671. Fix Version/s: 1.3.0 Resolution: Fixed > RBF: RouterRpcFairnessPolicyController suppo

[jira] [Resolved] (HDFS-16642) [SBN read] Moving selecting inputstream from JN in EditlogTailer out of FSNLock

2022-08-03 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16642. Fix Version/s: 3.4.0 Resolution: Fixed > [SBN read] Moving selecting inputstream from JN

[jira] [Resolved] (HDFS-16648) Normalize the usage of debug logs in NameNode

2022-08-06 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16648. Fix Version/s: 3.4.0 Resolution: Fixed > Normalize the usage of debug logs in NameN

[jira] [Resolved] (HDFS-16689) Standby NameNode crashes when transitioning to Active with in-progress tailer

2022-08-11 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16689. Fix Version/s: 3.4.0 Resolution: Fixed > Standby NameNode crashes when transitioning to Active w

[jira] [Reopened] (HDFS-16689) Standby NameNode crashes when transitioning to Active with in-progress tailer

2022-08-15 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei reopened HDFS-16689: > Standby NameNode crashes when transitioning to Active with in-progress tai

[jira] [Resolved] (HDFS-16738) Invalid CallerContext caused NullPointerException

2022-08-25 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16738. Fix Version/s: 3.4.0 Resolution: Fixed > Invalid CallerContext caused NullPointerExcept

[jira] [Resolved] (HDFS-16771) JN should tersely print logs about NewerTxnIdException

2022-09-24 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-16771. Resolution: Fixed > JN should tersely print logs about NewerTxnIdExcept

[jira] [Created] (HDFS-15604) Fix Typo for HdfsDataNodeAdminGuide doc

2020-09-28 Thread Hui Fei (Jira)
Hui Fei created HDFS-15604: -- Summary: Fix Typo for HdfsDataNodeAdminGuide doc Key: HDFS-15604 URL: https://issues.apache.org/jira/browse/HDFS-15604 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Reopened] (HDFS-14219) ConcurrentModificationException occurs in datanode occasionally

2020-10-13 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei reopened HDFS-14219: > ConcurrentModificationException occurs in datanode occasiona

[jira] [Resolved] (HDFS-14219) ConcurrentModificationException occurs in datanode occasionally

2020-10-13 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-14219. Resolution: Fixed > ConcurrentModificationException occurs in datanode occasiona

[jira] [Created] (HDFS-15652) Make block size from NNThroughputBenchmark configurable

2020-10-26 Thread Hui Fei (Jira)
Hui Fei created HDFS-15652: -- Summary: Make block size from NNThroughputBenchmark configurable Key: HDFS-15652 URL: https://issues.apache.org/jira/browse/HDFS-15652 Project: Hadoop HDFS Issue Type

[jira] [Reopened] (HDFS-15641) DataNode could meet deadlock if invoke refreshNameNode

2020-10-26 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei reopened HDFS-15641: > DataNode could meet deadlock if invoke refreshNameN

[jira] [Resolved] (HDFS-15641) DataNode could meet deadlock if invoke refreshNameNode

2020-10-26 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-15641. Resolution: Fixed > DataNode could meet deadlock if invoke refreshNameN

[jira] [Resolved] (HDFS-15667) Audit log record the unexpected allowed result when delete called

2020-11-09 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-15667. Fix Version/s: 3.4.0 Target Version/s: (was: 3.3.1, 3.4.0) Resolution: Fixed > Au

[jira] [Created] (HDFS-15698) Fix the typo of dfshealth.html after HDFS-15358

2020-11-27 Thread Hui Fei (Jira)
Hui Fei created HDFS-15698: -- Summary: Fix the typo of dfshealth.html after HDFS-15358 Key: HDFS-15698 URL: https://issues.apache.org/jira/browse/HDFS-15698 Project: Hadoop HDFS Issue Type: Bug

[jira] [Resolved] (HDFS-15698) Fix the typo of dfshealth.html after HDFS-15358

2020-11-27 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-15698. Fix Version/s: 3.4.0 Resolution: Fixed > Fix the typo of dfshealth.html after HDFS-15

[jira] [Resolved] (HDFS-14831) Downgrade Failed from 3.2.0 to 2.7 because of incompatible stringtable

2020-12-09 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-14831. Resolution: Fixed > Downgrade Failed from 3.2.0 to 2.7 because of incompatible stringta

[jira] [Resolved] (HDFS-15739) Missing Javadoc for a param in DFSNetworkTopology

2020-12-21 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-15739. Fix Version/s: 3.4.0 Resolution: Fixed > Missing Javadoc for a param in DFSNetworkTopol

[jira] [Resolved] (HDFS-15749) Make size of editPendingQ can be configurable

2020-12-27 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-15749. Fix Version/s: 3.4.0 Resolution: Fixed > Make size of editPendingQ can be configura

[jira] [Resolved] (HDFS-15819) Fix a codestyle issue for TestQuotaByStorageType

2021-02-04 Thread Hui Fei (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-15819. Fix Version/s: 3.4.0 Resolution: Fixed > Fix a codestyle issue for TestQuotaByStorageT

  1   2   >