Re: Discussion about NameNode Fine-grained locking

2024-02-20 Thread ZanderXu
Thank you everyone for reviewing this ticket. I think if there are no problems with the goal and the overall solution, we are ready to push this ticket forward and I will create some detailed sub-tasks for this ticket. I will split this project into three milestones to make this project cleaner f

Fwd: Discussion about NameNode Fine-grained locking

2024-04-23 Thread ZanderXu
Hi everyone All subtasks of the first phase of the FGL have been completed and I plan to merge them into the trunk and start the second phase based on the trunk. Here is the PR that used to merge the first phases into trunk: https://github.com/apache/hadoop/pull/6762 Here is the ticket: https://i

Re: Discussion about NameNode Fine-grained locking

2024-04-24 Thread ZanderXu
ecial thanks to @kokonguyen191 and @yuanboliu for their contributions. And you are also welcome to join us and complete it together. On Wed, 24 Apr 2024 at 14:54, ZanderXu wrote: > Hi everyone > > All subtasks of the first phase of the FGL have been completed and I plan > to merge th

Re: Discussion about NameNode Fine-grained locking

2024-04-26 Thread ZanderXu
ad/write throughput. >> >> I am happy to see the progress made on HDFS-17384. >> >> However, I still have some concerns, which roughly involve the following >> aspects: >> >> 1. While ZanderXu and Hui Fei have deep expertise in HDFS and are >> familia

Re: Discussion about NameNode Fine-grained locking

2024-04-29 Thread ZanderXu
s the problem, if that is the case then isn't that > the case with ErasureCoding policies or even ACLs or so? Can you elaborate > a bit on that. > > > Anyway, regarding the Phase-1. If you share (the perf numbers with proper > details + Impact on memory if any) for just phase

Re: Discussion about NameNode Fine-grained locking

2024-06-05 Thread ZanderXu
ing to discuss this? So that we can push it > forward quickly. Agreed with ZanderXu, it seems inefficient to discuss > details via email list. > > > Hui Fei 于2024年5月6日周一 23:50写道: > >> Thanks all >> >> Seems all concerns are related to the stage 2. We can

Re: Discussion about NameNode Fine-grained locking

2024-12-30 Thread ZanderXu
everyone a Happy New Year 2025! On Wed, 5 Jun 2024 at 16:17, ZanderXu wrote: > I plan to hold a meeting on 2024-06-06 from 3:00 PM - 4:00 PM to share the > FGL's motivations and some concerns in detail in Chinese. > > The doc is : NameNode Fine-Grained Locking Based On D

Re: Discussion about NameNode Fine-grained locking

2025-01-07 Thread ZanderXu
A0%E6%B5%A9%E5%8D%9A&uid=hfutzhanghb%40163.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fsmc804eb39b0e7885aa8801c3bb66e497d.jpg&items=%5B%22hfutzhanghb%40163.com%22%5D> >> >> Replied Message >> From haiyang hu >> Date 12/31/2024 23:08 >> To Ay

[VOTE] Merge HDFS-17384 to trunk

2025-01-13 Thread ZanderXu
ff: +1 (binding) Looking forward to your feedback. Best regards, ZanderXu

[jira] [Created] (HDFS-16587) Allow configuring Handler number for the JournalNodeRpcServer

2022-05-21 Thread ZanderXu (Jira)
ZanderXu created HDFS-16587: --- Summary: Allow configuring Handler number for the JournalNodeRpcServer Key: HDFS-16587 URL: https://issues.apache.org/jira/browse/HDFS-16587 Project: Hadoop HDFS

[jira] [Created] (HDFS-16593) Correct inaccurate BlocksRemoved metric on DataNode side

2022-05-24 Thread ZanderXu (Jira)
ZanderXu created HDFS-16593: --- Summary: Correct inaccurate BlocksRemoved metric on DataNode side Key: HDFS-16593 URL: https://issues.apache.org/jira/browse/HDFS-16593 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-16596) Improve the processing capability of FsDatasetAsyncDiskService

2022-05-26 Thread ZanderXu (Jira)
ZanderXu created HDFS-16596: --- Summary: Improve the processing capability of FsDatasetAsyncDiskService Key: HDFS-16596 URL: https://issues.apache.org/jira/browse/HDFS-16596 Project: Hadoop HDFS

[jira] [Created] (HDFS-16598) All datanodes [DatanodeInfoWithStorage[127.0.0.1:57448,DS-1b5f7e33-a2bf-4edc-9122-a74c995a99f5,DISK]] are bad. Aborting...

2022-05-27 Thread ZanderXu (Jira)
ZanderXu created HDFS-16598: --- Summary: All datanodes [DatanodeInfoWithStorage[127.0.0.1:57448,DS-1b5f7e33-a2bf-4edc-9122-a74c995a99f5,DISK]] are bad. Aborting... Key: HDFS-16598 URL: https://issues.apache.org/jira

[jira] [Created] (HDFS-16600) Deadlock on DataNode

2022-05-27 Thread ZanderXu (Jira)
ZanderXu created HDFS-16600: --- Summary: Deadlock on DataNode Key: HDFS-16600 URL: https://issues.apache.org/jira/browse/HDFS-16600 Project: Hadoop HDFS Issue Type: Bug Reporter

[jira] [Created] (HDFS-16601) Failed to replace a bad datanode on the existing pipeline due to no more good datanodes being available to try

2022-05-27 Thread ZanderXu (Jira)
ZanderXu created HDFS-16601: --- Summary: Failed to replace a bad datanode on the existing pipeline due to no more good datanodes being available to try Key: HDFS-16601 URL: https://issues.apache.org/jira/browse/HDFS

[jira] [Created] (HDFS-16622) addRDBI in IncrementalBlockReportManager may remove the block with bigger GS.

2022-06-05 Thread ZanderXu (Jira)
ZanderXu created HDFS-16622: --- Summary: addRDBI in IncrementalBlockReportManager may remove the block with bigger GS. Key: HDFS-16622 URL: https://issues.apache.org/jira/browse/HDFS-16622 Project: Hadoop

[jira] [Created] (HDFS-16623) IllegalArgumentException in LifelineSender

2022-06-06 Thread ZanderXu (Jira)
ZanderXu created HDFS-16623: --- Summary: IllegalArgumentException in LifelineSender Key: HDFS-16623 URL: https://issues.apache.org/jira/browse/HDFS-16623 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-16641) [HDFS] Add RPC ReQueue Metrics

2022-06-23 Thread ZanderXu (Jira)
ZanderXu created HDFS-16641: --- Summary: [HDFS] Add RPC ReQueue Metrics Key: HDFS-16641 URL: https://issues.apache.org/jira/browse/HDFS-16641 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Created] (HDFS-16642) [HDFS] Moving selecting inputstream from JN in EditlogTailer out of FSNLock

2022-06-24 Thread ZanderXu (Jira)
ZanderXu created HDFS-16642: --- Summary: [HDFS] Moving selecting inputstream from JN in EditlogTailer out of FSNLock Key: HDFS-16642 URL: https://issues.apache.org/jira/browse/HDFS-16642 Project: Hadoop HDFS

[jira] [Created] (HDFS-16645) Multi inProgress segments caused "Invalid log manifest"

2022-06-30 Thread ZanderXu (Jira)
ZanderXu created HDFS-16645: --- Summary: Multi inProgress segments caused "Invalid log manifest" Key: HDFS-16645 URL: https://issues.apache.org/jira/browse/HDFS-16645 Project: Hadoop HDFS

[jira] [Created] (HDFS-16646) [RBF] Improved isolation for downstream name nodes. {Elastic}

2022-06-30 Thread ZanderXu (Jira)
ZanderXu created HDFS-16646: --- Summary: [RBF] Improved isolation for downstream name nodes. {Elastic} Key: HDFS-16646 URL: https://issues.apache.org/jira/browse/HDFS-16646 Project: Hadoop HDFS

[jira] [Created] (HDFS-16648) Normalize the usage of debug logs in NameNode

2022-07-01 Thread ZanderXu (Jira)
ZanderXu created HDFS-16648: --- Summary: Normalize the usage of debug logs in NameNode Key: HDFS-16648 URL: https://issues.apache.org/jira/browse/HDFS-16648 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-16658) BlockManager should output some logs when logEveryBlock is true.

2022-07-13 Thread ZanderXu (Jira)
ZanderXu created HDFS-16658: --- Summary: BlockManager should output some logs when logEveryBlock is true. Key: HDFS-16658 URL: https://issues.apache.org/jira/browse/HDFS-16658 Project: Hadoop HDFS

[jira] [Created] (HDFS-16659) JournalNode should throw CacheMissException when SinceTxId is more than HighestWrittenTxId

2022-07-13 Thread ZanderXu (Jira)
ZanderXu created HDFS-16659: --- Summary: JournalNode should throw CacheMissException when SinceTxId is more than HighestWrittenTxId Key: HDFS-16659 URL: https://issues.apache.org/jira/browse/HDFS-16659

[jira] [Created] (HDFS-16660) Improve Code With Lambda in IPCLoggerChannel class

2022-07-13 Thread ZanderXu (Jira)
ZanderXu created HDFS-16660: --- Summary: Improve Code With Lambda in IPCLoggerChannel class Key: HDFS-16660 URL: https://issues.apache.org/jira/browse/HDFS-16660 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-16661) Improve Code With Lambda in AsyncLoggerSet class

2022-07-14 Thread ZanderXu (Jira)
ZanderXu created HDFS-16661: --- Summary: Improve Code With Lambda in AsyncLoggerSet class Key: HDFS-16661 URL: https://issues.apache.org/jira/browse/HDFS-16661 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-16670) Improve Code With Lambda in EditLogTailer class

2022-07-20 Thread ZanderXu (Jira)
ZanderXu created HDFS-16670: --- Summary: Improve Code With Lambda in EditLogTailer class Key: HDFS-16670 URL: https://issues.apache.org/jira/browse/HDFS-16670 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-16671) RBF: RouterRpcFairnessPolicyController supports configurable permit acquire timeout

2022-07-20 Thread ZanderXu (Jira)
ZanderXu created HDFS-16671: --- Summary: RBF: RouterRpcFairnessPolicyController supports configurable permit acquire timeout Key: HDFS-16671 URL: https://issues.apache.org/jira/browse/HDFS-16671 Project

[jira] [Created] (HDFS-16678) RBF supports disable getNodeUsage() in RBFMetrics

2022-07-21 Thread ZanderXu (Jira)
ZanderXu created HDFS-16678: --- Summary: RBF supports disable getNodeUsage() in RBFMetrics Key: HDFS-16678 URL: https://issues.apache.org/jira/browse/HDFS-16678 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-16689) Standby NameNode crashes when transitioning to Active with in-progress tailer

2022-07-25 Thread ZanderXu (Jira)
ZanderXu created HDFS-16689: --- Summary: Standby NameNode crashes when transitioning to Active with in-progress tailer Key: HDFS-16689 URL: https://issues.apache.org/jira/browse/HDFS-16689 Project: Hadoop

[jira] [Created] (HDFS-16692) Add detailed scope info in NotEnoughReplicas Reason logs.

2022-07-26 Thread ZanderXu (Jira)
ZanderXu created HDFS-16692: --- Summary: Add detailed scope info in NotEnoughReplicas Reason logs. Key: HDFS-16692 URL: https://issues.apache.org/jira/browse/HDFS-16692 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-16695) Improve Code With Lambda in hadoop-hdfs module

2022-07-27 Thread ZanderXu (Jira)
ZanderXu created HDFS-16695: --- Summary: Improve Code With Lambda in hadoop-hdfs module Key: HDFS-16695 URL: https://issues.apache.org/jira/browse/HDFS-16695 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-16696) NameNode supports a new MsyncRPCServer to reduce the latency of msync() rpc

2022-07-27 Thread ZanderXu (Jira)
ZanderXu created HDFS-16696: --- Summary: NameNode supports a new MsyncRPCServer to reduce the latency of msync() rpc Key: HDFS-16696 URL: https://issues.apache.org/jira/browse/HDFS-16696 Project: Hadoop HDFS

[jira] [Created] (HDFS-16698) Add a metric to sense possible MaxDirectoryItemsExceededException in time.

2022-07-27 Thread ZanderXu (Jira)
ZanderXu created HDFS-16698: --- Summary: Add a metric to sense possible MaxDirectoryItemsExceededException in time. Key: HDFS-16698 URL: https://issues.apache.org/jira/browse/HDFS-16698 Project: Hadoop HDFS

[jira] [Created] (HDFS-16703) Enable RPC Timeout for some protocols of NameNode.

2022-07-29 Thread ZanderXu (Jira)
ZanderXu created HDFS-16703: --- Summary: Enable RPC Timeout for some protocols of NameNode. Key: HDFS-16703 URL: https://issues.apache.org/jira/browse/HDFS-16703 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-16704) Datanode return empty response instead of NPE for GetVolumeInfo during restarting

2022-07-29 Thread ZanderXu (Jira)
ZanderXu created HDFS-16704: --- Summary: Datanode return empty response instead of NPE for GetVolumeInfo during restarting Key: HDFS-16704 URL: https://issues.apache.org/jira/browse/HDFS-16704 Project

[jira] [Created] (HDFS-16705) RBF supports healthMonitor timeout configurable and cache NN and client proxy in NamenodeHeartbeatService

2022-07-30 Thread ZanderXu (Jira)
ZanderXu created HDFS-16705: --- Summary: RBF supports healthMonitor timeout configurable and cache NN and client proxy in NamenodeHeartbeatService Key: HDFS-16705 URL: https://issues.apache.org/jira/browse/HDFS-16705

[jira] [Created] (HDFS-16709) Remove redundant cast in FSEditLogOp.class

2022-08-01 Thread ZanderXu (Jira)
ZanderXu created HDFS-16709: --- Summary: Remove redundant cast in FSEditLogOp.class Key: HDFS-16709 URL: https://issues.apache.org/jira/browse/HDFS-16709 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Created] (HDFS-16710) Remove redundant throw exceptions in org.apahce.hadoop.hdfs.server.namenode package

2022-08-01 Thread ZanderXu (Jira)
ZanderXu created HDFS-16710: --- Summary: Remove redundant throw exceptions in org.apahce.hadoop.hdfs.server.namenode package Key: HDFS-16710 URL: https://issues.apache.org/jira/browse/HDFS-16710 Project

[jira] [Resolved] (HDFS-16670) Improve Code With Lambda in EditLogTailer class

2022-08-01 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16670. - Resolution: Duplicate > Improve Code With Lambda in EditLogTailer cl

[jira] [Created] (HDFS-16712) Fix incorrect placeholder in DataNode.java

2022-08-02 Thread ZanderXu (Jira)
ZanderXu created HDFS-16712: --- Summary: Fix incorrect placeholder in DataNode.java Key: HDFS-16712 URL: https://issues.apache.org/jira/browse/HDFS-16712 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Created] (HDFS-16713) Improve Code with Lambda in org.apahce.hadoop.hdfs.server.namenode sub packages

2022-08-02 Thread ZanderXu (Jira)
ZanderXu created HDFS-16713: --- Summary: Improve Code with Lambda in org.apahce.hadoop.hdfs.server.namenode sub packages Key: HDFS-16713 URL: https://issues.apache.org/jira/browse/HDFS-16713 Project: Hadoop

[jira] [Created] (HDFS-16715) HostRestrictingAuthorizationFilter#matchRule should catch IOException

2022-08-03 Thread ZanderXu (Jira)
ZanderXu created HDFS-16715: --- Summary: HostRestrictingAuthorizationFilter#matchRule should catch IOException Key: HDFS-16715 URL: https://issues.apache.org/jira/browse/HDFS-16715 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-16715) HostRestrictingAuthorizationFilter#matchRule should catch IOException

2022-08-03 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16715. - Resolution: Invalid > HostRestrictingAuthorizationFilter#matchRule should catch IOExcept

[jira] [Created] (HDFS-16717) Replace NPE with IOException in DataNode.class

2022-08-04 Thread ZanderXu (Jira)
ZanderXu created HDFS-16717: --- Summary: Replace NPE with IOException in DataNode.class Key: HDFS-16717 URL: https://issues.apache.org/jira/browse/HDFS-16717 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-16718) Improve Code with Lambda in org.apahce.hadoop.hdfs.server.datanode packages

2022-08-05 Thread ZanderXu (Jira)
ZanderXu created HDFS-16718: --- Summary: Improve Code with Lambda in org.apahce.hadoop.hdfs.server.datanode packages Key: HDFS-16718 URL: https://issues.apache.org/jira/browse/HDFS-16718 Project: Hadoop HDFS

[jira] [Created] (HDFS-16719) Remove useless import in hadoop-hdfs module

2022-08-05 Thread ZanderXu (Jira)
ZanderXu created HDFS-16719: --- Summary: Remove useless import in hadoop-hdfs module Key: HDFS-16719 URL: https://issues.apache.org/jira/browse/HDFS-16719 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-16720) Import detailed class instead of import * in hadoop-hdfs moulde

2022-08-05 Thread ZanderXu (Jira)
ZanderXu created HDFS-16720: --- Summary: Import detailed class instead of import * in hadoop-hdfs moulde Key: HDFS-16720 URL: https://issues.apache.org/jira/browse/HDFS-16720 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-16719) Remove unused import in hadoop-hdfs module

2022-08-05 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16719. - Resolution: Invalid > Remove unused import in hadoop-hdfs mod

[jira] [Resolved] (HDFS-16720) Import detailed class instead of import * in hadoop-hdfs moulde

2022-08-05 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16720. - Resolution: Invalid > Import detailed class instead of import * in hadoop-hdfs mou

[jira] [Created] (HDFS-16723) Replace incorrect SafeModeException with StandbyException in RouterRpcServer.class

2022-08-07 Thread ZanderXu (Jira)
ZanderXu created HDFS-16723: --- Summary: Replace incorrect SafeModeException with StandbyException in RouterRpcServer.class Key: HDFS-16723 URL: https://issues.apache.org/jira/browse/HDFS-16723 Project

[jira] [Resolved] (HDFS-15310) RBF: Not proxy client's clientId and callId caused RetryCache invalid in NameNode.

2022-08-08 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-15310. - Resolution: Duplicate > RBF: Not proxy client's clientId and callId caused RetryCache in

[jira] [Created] (HDFS-16724) RBF should support get the information about ancestor mount points

2022-08-08 Thread ZanderXu (Jira)
ZanderXu created HDFS-16724: --- Summary: RBF should support get the information about ancestor mount points Key: HDFS-16724 URL: https://issues.apache.org/jira/browse/HDFS-16724 Project: Hadoop HDFS

[jira] [Created] (HDFS-16728) RBF throw IndexOutOfBoundsException with disableNameServices

2022-08-11 Thread ZanderXu (Jira)
ZanderXu created HDFS-16728: --- Summary: RBF throw IndexOutOfBoundsException with disableNameServices Key: HDFS-16728 URL: https://issues.apache.org/jira/browse/HDFS-16728 Project: Hadoop HDFS

[jira] [Created] (HDFS-16734) RBF: fix some bugs when handling getContentSummary RPC

2022-08-19 Thread ZanderXu (Jira)
ZanderXu created HDFS-16734: --- Summary: RBF: fix some bugs when handling getContentSummary RPC Key: HDFS-16734 URL: https://issues.apache.org/jira/browse/HDFS-16734 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-16737) Fix number of threads in FsDatasetAsyncDiskService#addExecutorForVolume

2022-08-22 Thread ZanderXu (Jira)
ZanderXu created HDFS-16737: --- Summary: Fix number of threads in FsDatasetAsyncDiskService#addExecutorForVolume Key: HDFS-16737 URL: https://issues.apache.org/jira/browse/HDFS-16737 Project: Hadoop HDFS

[jira] [Created] (HDFS-16738) Invalid CallerContext caused NullPointerException

2022-08-22 Thread ZanderXu (Jira)
ZanderXu created HDFS-16738: --- Summary: Invalid CallerContext caused NullPointerException Key: HDFS-16738 URL: https://issues.apache.org/jira/browse/HDFS-16738 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-16748) DFSClient should diff the writing files with namespace Id and iNodeId.

2022-08-27 Thread ZanderXu (Jira)
ZanderXu created HDFS-16748: --- Summary: DFSClient should diff the writing files with namespace Id and iNodeId. Key: HDFS-16748 URL: https://issues.apache.org/jira/browse/HDFS-16748 Project: Hadoop HDFS

[jira] [Created] (HDFS-16750) NameNode should use NameNode.getRemoteUser() to log audit event to avoid possible NPE

2022-08-29 Thread ZanderXu (Jira)
ZanderXu created HDFS-16750: --- Summary: NameNode should use NameNode.getRemoteUser() to log audit event to avoid possible NPE Key: HDFS-16750 URL: https://issues.apache.org/jira/browse/HDFS-16750 Project

[jira] [Created] (HDFS-16754) UnderConstruct file with missingBlock should be able to recover lease

2022-08-31 Thread ZanderXu (Jira)
ZanderXu created HDFS-16754: --- Summary: UnderConstruct file with missingBlock should be able to recover lease Key: HDFS-16754 URL: https://issues.apache.org/jira/browse/HDFS-16754 Project: Hadoop HDFS

[jira] [Created] (HDFS-16756) RBF proxies the client's user by the login user to enable CacheEntry

2022-09-05 Thread ZanderXu (Jira)
ZanderXu created HDFS-16756: --- Summary: RBF proxies the client's user by the login user to enable CacheEntry Key: HDFS-16756 URL: https://issues.apache.org/jira/browse/HDFS-16756 Project: Hadoop

[jira] [Created] (HDFS-16757) Add a new method copyBlockCrossNamespace to DataNode

2022-09-06 Thread ZanderXu (Jira)
ZanderXu created HDFS-16757: --- Summary: Add a new method copyBlockCrossNamespace to DataNode Key: HDFS-16757 URL: https://issues.apache.org/jira/browse/HDFS-16757 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-16758) Add an optional RetriableFileFastCopyCommand to Distcp

2022-09-06 Thread ZanderXu (Jira)
ZanderXu created HDFS-16758: --- Summary: Add an optional RetriableFileFastCopyCommand to Distcp Key: HDFS-16758 URL: https://issues.apache.org/jira/browse/HDFS-16758 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-16759) RetriableFileFastCopyCommand supports migrating EC files

2022-09-06 Thread ZanderXu (Jira)
ZanderXu created HDFS-16759: --- Summary: RetriableFileFastCopyCommand supports migrating EC files Key: HDFS-16759 URL: https://issues.apache.org/jira/browse/HDFS-16759 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-16760) Support a block level inputFormat for RetriableFileFastCopyCommand

2022-09-06 Thread ZanderXu (Jira)
ZanderXu created HDFS-16760: --- Summary: Support a block level inputFormat for RetriableFileFastCopyCommand Key: HDFS-16760 URL: https://issues.apache.org/jira/browse/HDFS-16760 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-16641) [HDFS] Add RPC ReQueue Metrics

2022-09-07 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16641. - Resolution: Duplicate > [HDFS] Add RPC ReQueue Metr

[jira] [Created] (HDFS-16764) ObserverNamenode handles addBlock rpc and throws a FileNotFoundException

2022-09-07 Thread ZanderXu (Jira)
ZanderXu created HDFS-16764: --- Summary: ObserverNamenode handles addBlock rpc and throws a FileNotFoundException Key: HDFS-16764 URL: https://issues.apache.org/jira/browse/HDFS-16764 Project: Hadoop HDFS

[jira] [Created] (HDFS-16771) JN should tersely print logs about NewerTxnIdException

2022-09-13 Thread ZanderXu (Jira)
ZanderXu created HDFS-16771: --- Summary: JN should tersely print logs about NewerTxnIdException Key: HDFS-16771 URL: https://issues.apache.org/jira/browse/HDFS-16771 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-16772) refreshHostsReader should use the new configuration

2022-09-14 Thread ZanderXu (Jira)
ZanderXu created HDFS-16772: --- Summary: refreshHostsReader should use the new configuration Key: HDFS-16772 URL: https://issues.apache.org/jira/browse/HDFS-16772 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-16779) Add ErasureCodingPolicy information to the response description for GETFILESTATUS in WebHDFS.md

2022-09-21 Thread ZanderXu (Jira)
ZanderXu created HDFS-16779: --- Summary: Add ErasureCodingPolicy information to the response description for GETFILESTATUS in WebHDFS.md Key: HDFS-16779 URL: https://issues.apache.org/jira/browse/HDFS-16779

[jira] [Created] (HDFS-16780) Add ErasureCodePolicy information to the response of GET_BLOCK_LOCATIONS in NamenodeWebHdfsMethods

2022-09-21 Thread ZanderXu (Jira)
ZanderXu created HDFS-16780: --- Summary: Add ErasureCodePolicy information to the response of GET_BLOCK_LOCATIONS in NamenodeWebHdfsMethods Key: HDFS-16780 URL: https://issues.apache.org/jira/browse/HDFS-16780

[jira] [Created] (HDFS-16783) Remove the redundant lock in deepCopyReplica

2022-09-27 Thread ZanderXu (Jira)
ZanderXu created HDFS-16783: --- Summary: Remove the redundant lock in deepCopyReplica Key: HDFS-16783 URL: https://issues.apache.org/jira/browse/HDFS-16783 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-16785) DataNode hold BP write lock to scan disk

2022-09-28 Thread ZanderXu (Jira)
ZanderXu created HDFS-16785: --- Summary: DataNode hold BP write lock to scan disk Key: HDFS-16785 URL: https://issues.apache.org/jira/browse/HDFS-16785 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Created] (HDFS-16787) Remove redundant lock in DataSetLockManager#removeLock in datanode.

2022-09-28 Thread ZanderXu (Jira)
ZanderXu created HDFS-16787: --- Summary: Remove redundant lock in DataSetLockManager#removeLock in datanode. Key: HDFS-16787 URL: https://issues.apache.org/jira/browse/HDFS-16787 Project: Hadoop HDFS

[jira] [Created] (HDFS-16793) ObserverNameNode fails to select streaming inputStream with a timeout exception

2022-10-04 Thread ZanderXu (Jira)
ZanderXu created HDFS-16793: --- Summary: ObserverNameNode fails to select streaming inputStream with a timeout exception Key: HDFS-16793 URL: https://issues.apache.org/jira/browse/HDFS-16793 Project: Hadoop

[jira] [Resolved] (HDFS-16596) Improve the processing capability of FsDatasetAsyncDiskService

2022-10-05 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16596. - Resolution: Duplicate > Improve the processing capability of FsDatasetAsyncDiskServ

[jira] [Created] (HDFS-16798) SerialNumberMap should decrease the current if the old already exist

2022-10-08 Thread ZanderXu (Jira)
ZanderXu created HDFS-16798: --- Summary: SerialNumberMap should decrease the current if the old already exist Key: HDFS-16798 URL: https://issues.apache.org/jira/browse/HDFS-16798 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-16774) Improve async delete replica on datanode

2022-10-11 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16774. - Fix Version/s: 3.4.0 Resolution: Fixed > Improve async delete replica on datan

[jira] [Created] (HDFS-16804) AddVolume contains a race condition with shutdown block pool

2022-10-14 Thread ZanderXu (Jira)
ZanderXu created HDFS-16804: --- Summary: AddVolume contains a race condition with shutdown block pool Key: HDFS-16804 URL: https://issues.apache.org/jira/browse/HDFS-16804 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-16803) Improve some annotations in hdfs module

2022-10-19 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16803. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Improve some annotations

[jira] [Resolved] (HDFS-16817) Remove useless DataNode lock related configuration

2022-10-27 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16817. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Remove useless DataNode l

[jira] [Resolved] (HDFS-16802) Print options when accessing ClientProtocol#rename2()

2022-10-28 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16802. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Print options when access

[jira] [Created] (HDFS-16826) [RBF SBN] ConnectionManager should advance the client stateId for every request

2022-10-28 Thread ZanderXu (Jira)
ZanderXu created HDFS-16826: --- Summary: [RBF SBN] ConnectionManager should advance the client stateId for every request Key: HDFS-16826 URL: https://issues.apache.org/jira/browse/HDFS-16826 Project: Hadoop

[jira] [Created] (HDFS-16827) [RBF SBN] RouterStateIdContext shouldn't update the ResponseState if client doesn't use ObserverReadProxyProvider

2022-10-28 Thread ZanderXu (Jira)
ZanderXu created HDFS-16827: --- Summary: [RBF SBN] RouterStateIdContext shouldn't update the ResponseState if client doesn't use ObserverReadProxyProvider Key: HDFS-16827 URL: https://issues.apache.org/jira/b

[jira] [Created] (HDFS-16831) [RBF SBN] GetNamenodesForNameserviceId should shuffle Observer NameNodes every time

2022-11-01 Thread ZanderXu (Jira)
ZanderXu created HDFS-16831: --- Summary: [RBF SBN] GetNamenodesForNameserviceId should shuffle Observer NameNodes every time Key: HDFS-16831 URL: https://issues.apache.org/jira/browse/HDFS-16831 Project

[jira] [Created] (HDFS-16837) [RBF SBN] ClientGSIContext should merge RouterFederatedStates to get the max state id for each namespace

2022-11-10 Thread ZanderXu (Jira)
ZanderXu created HDFS-16837: --- Summary: [RBF SBN] ClientGSIContext should merge RouterFederatedStates to get the max state id for each namespace Key: HDFS-16837 URL: https://issues.apache.org/jira/browse/HDFS-16837

[jira] [Created] (HDFS-16838) Fix NPE in testAddRplicaProcessorForAddingReplicaInMap

2022-11-10 Thread ZanderXu (Jira)
ZanderXu created HDFS-16838: --- Summary: Fix NPE in testAddRplicaProcessorForAddingReplicaInMap Key: HDFS-16838 URL: https://issues.apache.org/jira/browse/HDFS-16838 Project: Hadoop HDFS Issue Type

[jira] [Resolved] (HDFS-16834) PoolAlignmentContext should not max poolLocalStateId with sharedGlobalStateId when sending requests to the namenode.

2022-11-10 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16834. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > PoolAlignmentContext sho

[jira] [Resolved] (HDFS-16813) Remove parameter validation logic such as dfs.namenode.decommission.blocks.per.interval in DatanodeAdminManager#activate

2022-11-22 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16813. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Remove parameter validat

[jira] [Created] (HDFS-16853) The UT TestLeaseRecovery2#testHardLeaseRecoveryAfterNameNodeRestart failed because HADOOP-18324

2022-11-23 Thread ZanderXu (Jira)
ZanderXu created HDFS-16853: --- Summary: The UT TestLeaseRecovery2#testHardLeaseRecoveryAfterNameNodeRestart failed because HADOOP-18324 Key: HDFS-16853 URL: https://issues.apache.org/jira/browse/HDFS-16853

[jira] [Resolved] (HDFS-16841) Enhance the function of DebugAdmin#VerifyECCommand

2022-11-23 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16841. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Enhance the function

[jira] [Resolved] (HDFS-16826) [RBF SBN] ConnectionManager should advance the client stateId for every request

2022-11-24 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16826. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > [RBF SBN] ConnectionMana

[jira] [Resolved] (HDFS-16779) Add ErasureCodingPolicy information to the response description for GETFILESTATUS in WebHDFS.md

2022-11-24 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16779. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Add ErasureCodingPol

[jira] [Resolved] (HDFS-16837) [RBF SBN] ClientGSIContext should merge RouterFederatedStates to get the max state id for each namespace

2022-12-05 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16837. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > [RBF SBN] ClientGSICont

[jira] [Created] (HDFS-16865) The source path is always / after RBF proxied the complete, addBlock and getAdditionalDatanode RPC.

2022-12-09 Thread ZanderXu (Jira)
ZanderXu created HDFS-16865: --- Summary: The source path is always / after RBF proxied the complete, addBlock and getAdditionalDatanode RPC. Key: HDFS-16865 URL: https://issues.apache.org/jira/browse/HDFS-16865

[jira] [Resolved] (HDFS-16831) [RBF SBN] GetNamenodesForNameserviceId should shuffle Observer NameNodes every time

2022-12-22 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16831. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > [RBF

[jira] [Resolved] (HDFS-16879) EC : Fsck -blockId shows number of redundant internal block replicas for EC Blocks

2023-01-03 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16879. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > EC : Fsck -blockId sh

[jira] [Resolved] (HDFS-16910) Fix incorrectly initializing RandomAccessFile caused flush performance decreased for JN

2023-02-08 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16910. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Fix incorrectly initializ

[jira] [Created] (HDFS-16923) The getListing RPC will throw NPE if the path does not exist

2023-02-15 Thread ZanderXu (Jira)
ZanderXu created HDFS-16923: --- Summary: The getListing RPC will throw NPE if the path does not exist Key: HDFS-16923 URL: https://issues.apache.org/jira/browse/HDFS-16923 Project: Hadoop HDFS

[jira] [Created] (HDFS-16933) A race in SerialNumberMap will cause wrong ownership

2023-02-24 Thread ZanderXu (Jira)
ZanderXu created HDFS-16933: --- Summary: A race in SerialNumberMap will cause wrong ownership Key: HDFS-16933 URL: https://issues.apache.org/jira/browse/HDFS-16933 Project: Hadoop HDFS Issue Type

  1   2   >