[VOTE] Merge HDFS-17384 to trunk

2025-01-13 Thread ZanderXu
ff: +1 (binding) Looking forward to your feedback. Best regards, ZanderXu

Re: Discussion about NameNode Fine-grained locking

2025-01-07 Thread ZanderXu
A0%E6%B5%A9%E5%8D%9A&uid=hfutzhanghb%40163.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fsmc804eb39b0e7885aa8801c3bb66e497d.jpg&items=%5B%22hfutzhanghb%40163.com%22%5D> >> >> Replied Message >> From haiyang hu >> Date 12/31/2024 23:08 >> To Ay

Re: Discussion about NameNode Fine-grained locking

2024-12-30 Thread ZanderXu
everyone a Happy New Year 2025! On Wed, 5 Jun 2024 at 16:17, ZanderXu wrote: > I plan to hold a meeting on 2024-06-06 from 3:00 PM - 4:00 PM to share the > FGL's motivations and some concerns in detail in Chinese. > > The doc is : NameNode Fine-Grained Locking Based On D

[jira] [Created] (HDFS-17701) [FGL] Fix some javadocs

2024-12-29 Thread ZanderXu (Jira)
ZanderXu created HDFS-17701: --- Summary: [FGL] Fix some javadocs Key: HDFS-17701 URL: https://issues.apache.org/jira/browse/HDFS-17701 Project: Hadoop HDFS Issue Type: Sub-task Reporter

[jira] [Created] (HDFS-17697) [FGL] Fix javadoc for FGL Phase I

2024-12-26 Thread ZanderXu (Jira)
ZanderXu created HDFS-17697: --- Summary: [FGL] Fix javadoc for FGL Phase I Key: HDFS-17697 URL: https://issues.apache.org/jira/browse/HDFS-17697 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Resolved] (HDFS-17692) [FGL] Fix bugs in `getContentSummary`

2024-12-25 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17692. - Hadoop Flags: Reviewed Resolution: Fixed > [FGL] Fix bugs in `getContentSumm

[jira] [Resolved] (HDFS-17482) [FGL] Solutions for QuotaUpdated when completed block

2024-12-25 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17482. - Resolution: Fixed > [FGL] Solutions for QuotaUpdated when completed bl

[jira] [Resolved] (HDFS-17480) [FGL] Solutions for GetListing RPC

2024-12-25 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17480. - Resolution: Fixed > [FGL] Solutions for GetListing

[jira] [Resolved] (HDFS-17507) [FGL] Solutions for Symlinks

2024-12-25 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17507. - Resolution: Fixed > [FGL] Solutions for Symlinks > > >

[jira] [Resolved] (HDFS-17505) [FGL] Solutions for StoragePolicy

2024-12-25 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17505. - Resolution: Fixed > [FGL] Solutions for StoragePol

[jira] [Resolved] (HDFS-17473) [FGL] Solutions for Quota

2024-12-25 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17473. - Resolution: Fixed > [FGL] Solutions for Quota > - > >

[jira] [Created] (HDFS-17692) [FGL] Fix bugs in `getContentSummary`

2024-12-18 Thread ZanderXu (Jira)
ZanderXu created HDFS-17692: --- Summary: [FGL] Fix bugs in `getContentSummary` Key: HDFS-17692 URL: https://issues.apache.org/jira/browse/HDFS-17692 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-17691) [FGL] Move FSNamesystemLockMode to org.apache.hadoop.hdfs.util package

2024-12-17 Thread ZanderXu (Jira)
ZanderXu created HDFS-17691: --- Summary: [FGL] Move FSNamesystemLockMode to org.apache.hadoop.hdfs.util package Key: HDFS-17691 URL: https://issues.apache.org/jira/browse/HDFS-17691 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-17634) RBF: Web UI missing DN last block report

2024-10-22 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17634. - Hadoop Flags: Reviewed Resolution: Fixed > RBF: Web UI missing DN last block rep

[jira] [Created] (HDFS-17603) [FGL] Abstract a LockManager to manage locks

2024-08-14 Thread ZanderXu (Jira)
ZanderXu created HDFS-17603: --- Summary: [FGL] Abstract a LockManager to manage locks Key: HDFS-17603 URL: https://issues.apache.org/jira/browse/HDFS-17603 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Resolved] (HDFS-17539) TestFileChecksum should not spin up a MiniDFSCluster for every test

2024-06-11 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17539. - Fix Version/s: 3.5.0 Hadoop Flags: Reviewed Resolution: Fixed > TestFileChecksum should

Re: Discussion about NameNode Fine-grained locking

2024-06-05 Thread ZanderXu
ing to discuss this? So that we can push it > forward quickly. Agreed with ZanderXu, it seems inefficient to discuss > details via email list. > > > Hui Fei 于2024年5月6日周一 23:50写道: > >> Thanks all >> >> Seems all concerns are related to the stage 2. We can

[jira] [Resolved] (HDFS-17532) RBF: Allow router state store cache update to overwrite and delete in parallel

2024-05-27 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17532. - Hadoop Flags: Reviewed Resolution: Fixed > RBF: Allow router state store cache update to overwr

[jira] [Created] (HDFS-17536) RBF: Format safe-mode related logic and fix a race

2024-05-24 Thread ZanderXu (Jira)
ZanderXu created HDFS-17536: --- Summary: RBF: Format safe-mode related logic and fix a race Key: HDFS-17536 URL: https://issues.apache.org/jira/browse/HDFS-17536 Project: Hadoop HDFS Issue Type

[jira] [Resolved] (HDFS-17529) RBF: Improve router state store cache entry deletion

2024-05-23 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17529. - Fix Version/s: 3.5.0 Hadoop Flags: Reviewed Resolution: Fixed > RBF: Improve router st

[jira] [Resolved] (HDFS-17509) RBF: Fix ClientProtocol.concat will throw NPE if tgr is a empty file.

2024-05-16 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17509. - Fix Version/s: 3.5.0 Resolution: Fixed > RBF: Fix ClientProtocol.concat will throw NPE if tgr

[jira] [Resolved] (HDFS-17488) DN can fail IBRs with NPE when a volume is removed

2024-05-11 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17488. - Fix Version/s: 3.5.0 Resolution: Fixed > DN can fail IBRs with NPE when a volume is remo

[jira] [Created] (HDFS-17520) TestDFSAdmin.testAllDatanodesReconfig and TestDFSAdmin.testDecommissionDataNodesReconfig failed

2024-05-11 Thread ZanderXu (Jira)
ZanderXu created HDFS-17520: --- Summary: TestDFSAdmin.testAllDatanodesReconfig and TestDFSAdmin.testDecommissionDataNodesReconfig failed Key: HDFS-17520 URL: https://issues.apache.org/jira/browse/HDFS-17520

[jira] [Resolved] (HDFS-17503) Unreleased volume references because of OOM

2024-05-09 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17503. - Fix Version/s: 3.5.0 Hadoop Flags: Reviewed Resolution: Fixed > Unreleased volume referen

[jira] [Created] (HDFS-17517) [FGL] Abstract lock mode to cover all RPCs

2024-05-09 Thread ZanderXu (Jira)
ZanderXu created HDFS-17517: --- Summary: [FGL] Abstract lock mode to cover all RPCs Key: HDFS-17517 URL: https://issues.apache.org/jira/browse/HDFS-17517 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-17507) [FGL] Solutions for Symlinks

2024-04-29 Thread ZanderXu (Jira)
ZanderXu created HDFS-17507: --- Summary: [FGL] Solutions for Symlinks Key: HDFS-17507 URL: https://issues.apache.org/jira/browse/HDFS-17507 Project: Hadoop HDFS Issue Type: Sub-task

Re: Discussion about NameNode Fine-grained locking

2024-04-29 Thread ZanderXu
s the problem, if that is the case then isn't that > the case with ErasureCoding policies or even ACLs or so? Can you elaborate > a bit on that. > > > Anyway, regarding the Phase-1. If you share (the perf numbers with proper > details + Impact on memory if any) for just phase

[jira] [Created] (HDFS-17506) [FGL] Performance for phase 1

2024-04-29 Thread ZanderXu (Jira)
ZanderXu created HDFS-17506: --- Summary: [FGL] Performance for phase 1 Key: HDFS-17506 URL: https://issues.apache.org/jira/browse/HDFS-17506 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-17505) [FGL] Solutions for StoragePolicy

2024-04-29 Thread ZanderXu (Jira)
ZanderXu created HDFS-17505: --- Summary: [FGL] Solutions for StoragePolicy Key: HDFS-17505 URL: https://issues.apache.org/jira/browse/HDFS-17505 Project: Hadoop HDFS Issue Type: Sub-task

Re: Discussion about NameNode Fine-grained locking

2024-04-26 Thread ZanderXu
ad/write throughput. >> >> I am happy to see the progress made on HDFS-17384. >> >> However, I still have some concerns, which roughly involve the following >> aspects: >> >> 1. While ZanderXu and Hui Fei have deep expertise in HDFS and are >> familia

Re: Discussion about NameNode Fine-grained locking

2024-04-24 Thread ZanderXu
ecial thanks to @kokonguyen191 and @yuanboliu for their contributions. And you are also welcome to join us and complete it together. On Wed, 24 Apr 2024 at 14:54, ZanderXu wrote: > Hi everyone > > All subtasks of the first phase of the FGL have been completed and I plan > to merge th

Fwd: Discussion about NameNode Fine-grained locking

2024-04-23 Thread ZanderXu
Hi everyone All subtasks of the first phase of the FGL have been completed and I plan to merge them into the trunk and start the second phase based on the trunk. Here is the PR that used to merge the first phases into trunk: https://github.com/apache/hadoop/pull/6762 Here is the ticket: https://i

[jira] [Created] (HDFS-17497) Logic for committed blocks is mixed when computing file size

2024-04-23 Thread ZanderXu (Jira)
ZanderXu created HDFS-17497: --- Summary: Logic for committed blocks is mixed when computing file size Key: HDFS-17497 URL: https://issues.apache.org/jira/browse/HDFS-17497 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-17485) Fix SpotBug in RouterRpcServer.java

2024-04-22 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17485. - Resolution: Duplicate > Fix SpotBug in RouterRpcServer.j

[jira] [Created] (HDFS-17494) [FGL] GetFileInfo supports fine-grained locking

2024-04-22 Thread ZanderXu (Jira)
ZanderXu created HDFS-17494: --- Summary: [FGL] GetFileInfo supports fine-grained locking Key: HDFS-17494 URL: https://issues.apache.org/jira/browse/HDFS-17494 Project: Hadoop HDFS Issue Type: Sub

[jira] [Created] (HDFS-17493) [FGL] Make INodeMap thread safe

2024-04-22 Thread ZanderXu (Jira)
ZanderXu created HDFS-17493: --- Summary: [FGL] Make INodeMap thread safe Key: HDFS-17493 URL: https://issues.apache.org/jira/browse/HDFS-17493 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-17492) [FGL] Abstract a INodeLockManager to mange acquiring and releasing locks in the directory-tree

2024-04-22 Thread ZanderXu (Jira)
ZanderXu created HDFS-17492: --- Summary: [FGL] Abstract a INodeLockManager to mange acquiring and releasing locks in the directory-tree Key: HDFS-17492 URL: https://issues.apache.org/jira/browse/HDFS-17492

[jira] [Created] (HDFS-17491) [FGL] Make getFullPathName in INode.java thread safe

2024-04-22 Thread ZanderXu (Jira)
ZanderXu created HDFS-17491: --- Summary: [FGL] Make getFullPathName in INode.java thread safe Key: HDFS-17491 URL: https://issues.apache.org/jira/browse/HDFS-17491 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-17490) [FGL] Make INodesInPath closeable

2024-04-22 Thread ZanderXu (Jira)
ZanderXu created HDFS-17490: --- Summary: [FGL] Make INodesInPath closeable Key: HDFS-17490 URL: https://issues.apache.org/jira/browse/HDFS-17490 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-17489) [FGL] Implement a LockPool

2024-04-22 Thread ZanderXu (Jira)
ZanderXu created HDFS-17489: --- Summary: [FGL] Implement a LockPool Key: HDFS-17489 URL: https://issues.apache.org/jira/browse/HDFS-17489 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-17487) [FGL] Make rollEdits thread safe

2024-04-22 Thread ZanderXu (Jira)
ZanderXu created HDFS-17487: --- Summary: [FGL] Make rollEdits thread safe Key: HDFS-17487 URL: https://issues.apache.org/jira/browse/HDFS-17487 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-17485) Fix SpotBug in RouterRpcServer.java

2024-04-22 Thread ZanderXu (Jira)
ZanderXu created HDFS-17485: --- Summary: Fix SpotBug in RouterRpcServer.java Key: HDFS-17485 URL: https://issues.apache.org/jira/browse/HDFS-17485 Project: Hadoop HDFS Issue Type: Task

[jira] [Created] (HDFS-17482) [FGL] Update quotas when committing block

2024-04-18 Thread ZanderXu (Jira)
ZanderXu created HDFS-17482: --- Summary: [FGL] Update quotas when committing block Key: HDFS-17482 URL: https://issues.apache.org/jira/browse/HDFS-17482 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-17481) [FGL] ComputeFileSize in INodeFile should avoid BM lock

2024-04-18 Thread ZanderXu (Jira)
ZanderXu created HDFS-17481: --- Summary: [FGL] ComputeFileSize in INodeFile should avoid BM lock Key: HDFS-17481 URL: https://issues.apache.org/jira/browse/HDFS-17481 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-17480) [FGL] GetListing RPC supports fine-grained locking

2024-04-17 Thread ZanderXu (Jira)
ZanderXu created HDFS-17480: --- Summary: [FGL] GetListing RPC supports fine-grained locking Key: HDFS-17480 URL: https://issues.apache.org/jira/browse/HDFS-17480 Project: Hadoop HDFS Issue Type: Sub

[jira] [Created] (HDFS-17479) [FGL] Snapshot related operations still use global lock

2024-04-17 Thread ZanderXu (Jira)
ZanderXu created HDFS-17479: --- Summary: [FGL] Snapshot related operations still use global lock Key: HDFS-17479 URL: https://issues.apache.org/jira/browse/HDFS-17479 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-17473) [FGL] Make quota related operations thread-safe

2024-04-17 Thread ZanderXu (Jira)
ZanderXu created HDFS-17473: --- Summary: [FGL] Make quota related operations thread-safe Key: HDFS-17473 URL: https://issues.apache.org/jira/browse/HDFS-17473 Project: Hadoop HDFS Issue Type: Sub

[jira] [Created] (HDFS-17472) [FGL] gcDeletedSnapshot and getDelegationToken support FGL

2024-04-16 Thread ZanderXu (Jira)
ZanderXu created HDFS-17472: --- Summary: [FGL] gcDeletedSnapshot and getDelegationToken support FGL Key: HDFS-17472 URL: https://issues.apache.org/jira/browse/HDFS-17472 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-17459) [FGL] Summarize this feature

2024-04-09 Thread ZanderXu (Jira)
ZanderXu created HDFS-17459: --- Summary: [FGL] Summarize this feature Key: HDFS-17459 URL: https://issues.apache.org/jira/browse/HDFS-17459 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-17457) [FGL] UTs support fine-grained locking

2024-04-07 Thread ZanderXu (Jira)
ZanderXu created HDFS-17457: --- Summary: [FGL] UTs support fine-grained locking Key: HDFS-17457 URL: https://issues.apache.org/jira/browse/HDFS-17457 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-17445) [FGL] All remaining operations support fine-grained locking

2024-03-28 Thread ZanderXu (Jira)
ZanderXu created HDFS-17445: --- Summary: [FGL] All remaining operations support fine-grained locking Key: HDFS-17445 URL: https://issues.apache.org/jira/browse/HDFS-17445 Project: Hadoop HDFS Issue

[jira] [Resolved] (HDFS-17440) [FGL] Rebase the development branch on the trunk branch

2024-03-25 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17440. - Resolution: Fixed > [FGL] Rebase the development branch on the trunk bra

[jira] [Created] (HDFS-17440) [FGL] Rebase the development branch on the trunk branch

2024-03-25 Thread ZanderXu (Jira)
ZanderXu created HDFS-17440: --- Summary: [FGL] Rebase the development branch on the trunk branch Key: HDFS-17440 URL: https://issues.apache.org/jira/browse/HDFS-17440 Project: Hadoop HDFS Issue Type

[jira] [Resolved] (HDFS-17428) [FGL] StoragePolicySatisfyManager supports fine-grained lock

2024-03-21 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17428. - Resolution: Won't Do > [FGL] StoragePolicySatisfyManager supports fine-grai

[jira] [Resolved] (HDFS-17419) [FGL] CacheReplicationMonitor supports fine-grained lock

2024-03-21 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17419. - Resolution: Duplicate > [FGL] CacheReplicationMonitor supports fine-grained l

[jira] [Resolved] (HDFS-17418) [FGL] DatanodeAdminMonitor supports fine-grained locking

2024-03-21 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17418. - Resolution: Duplicate > [FGL] DatanodeAdminMonitor supports fine-grained lock

[jira] [Resolved] (HDFS-17426) Remove Invalid FileSystemECReadStats logic in DFSInputStream

2024-03-20 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17426. - Resolution: Fixed > Remove Invalid FileSystemECReadStats logic in DFSInputStr

[jira] [Created] (HDFS-17428) [FGL] StoragePolicySatisfyManager supports fine-grained lock

2024-03-18 Thread ZanderXu (Jira)
ZanderXu created HDFS-17428: --- Summary: [FGL] StoragePolicySatisfyManager supports fine-grained lock Key: HDFS-17428 URL: https://issues.apache.org/jira/browse/HDFS-17428 Project: Hadoop HDFS

[jira] [Created] (HDFS-17425) [FGL] Reencryption supports fine-grained lock

2024-03-12 Thread ZanderXu (Jira)
ZanderXu created HDFS-17425: --- Summary: [FGL] Reencryption supports fine-grained lock Key: HDFS-17425 URL: https://issues.apache.org/jira/browse/HDFS-17425 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-17424) [FGL] DelegationTokenSecretManager supports fine-grained lock

2024-03-12 Thread ZanderXu (Jira)
ZanderXu created HDFS-17424: --- Summary: [FGL] DelegationTokenSecretManager supports fine-grained lock Key: HDFS-17424 URL: https://issues.apache.org/jira/browse/HDFS-17424 Project: Hadoop HDFS

[jira] [Created] (HDFS-17423) [FGL] BlockManagerSafeMode supports fine-grained lock

2024-03-11 Thread ZanderXu (Jira)
ZanderXu created HDFS-17423: --- Summary: [FGL] BlockManagerSafeMode supports fine-grained lock Key: HDFS-17423 URL: https://issues.apache.org/jira/browse/HDFS-17423 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-17420) [FGL] FSEditLogLoader supports fine-grained lock

2024-03-07 Thread ZanderXu (Jira)
ZanderXu created HDFS-17420: --- Summary: [FGL] FSEditLogLoader supports fine-grained lock Key: HDFS-17420 URL: https://issues.apache.org/jira/browse/HDFS-17420 Project: Hadoop HDFS Issue Type: Sub

[jira] [Created] (HDFS-17419) [FGL] CacheReplicationMonitor supports fine-grained lock

2024-03-07 Thread ZanderXu (Jira)
ZanderXu created HDFS-17419: --- Summary: [FGL] CacheReplicationMonitor supports fine-grained lock Key: HDFS-17419 URL: https://issues.apache.org/jira/browse/HDFS-17419 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-17418) [FGL] DatanodeAdminMonitor supports fine-grained locking

2024-03-07 Thread ZanderXu (Jira)
ZanderXu created HDFS-17418: --- Summary: [FGL] DatanodeAdminMonitor supports fine-grained locking Key: HDFS-17418 URL: https://issues.apache.org/jira/browse/HDFS-17418 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-17417) [FGL] Monitor in HeartbeatManager supports fine-grained lock

2024-03-07 Thread ZanderXu (Jira)
ZanderXu created HDFS-17417: --- Summary: [FGL] Monitor in HeartbeatManager supports fine-grained lock Key: HDFS-17417 URL: https://issues.apache.org/jira/browse/HDFS-17417 Project: Hadoop HDFS

[jira] [Created] (HDFS-17416) [FGL] Monitor threads in BlockManager.class support fine-grained lock

2024-03-07 Thread ZanderXu (Jira)
ZanderXu created HDFS-17416: --- Summary: [FGL] Monitor threads in BlockManager.class support fine-grained lock Key: HDFS-17416 URL: https://issues.apache.org/jira/browse/HDFS-17416 Project: Hadoop HDFS

[jira] [Created] (HDFS-17415) [FGL] RPCs in NamenodeProtocol support fine-grained lock

2024-03-07 Thread ZanderXu (Jira)
ZanderXu created HDFS-17415: --- Summary: [FGL] RPCs in NamenodeProtocol support fine-grained lock Key: HDFS-17415 URL: https://issues.apache.org/jira/browse/HDFS-17415 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-17414) [FGL] RPCs in DatanodeProtocol support fine-grained lock

2024-03-07 Thread ZanderXu (Jira)
ZanderXu created HDFS-17414: --- Summary: [FGL] RPCs in DatanodeProtocol support fine-grained lock Key: HDFS-17414 URL: https://issues.apache.org/jira/browse/HDFS-17414 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-17413) [FGL] Client RPCs involving Cache supports fine-grained lock

2024-03-07 Thread ZanderXu (Jira)
ZanderXu created HDFS-17413: --- Summary: [FGL] Client RPCs involving Cache supports fine-grained lock Key: HDFS-17413 URL: https://issues.apache.org/jira/browse/HDFS-17413 Project: Hadoop HDFS

[jira] [Created] (HDFS-17412) [FGL] Client RPCs involving maintenance supports fine-grained lock

2024-03-07 Thread ZanderXu (Jira)
ZanderXu created HDFS-17412: --- Summary: [FGL] Client RPCs involving maintenance supports fine-grained lock Key: HDFS-17412 URL: https://issues.apache.org/jira/browse/HDFS-17412 Project: Hadoop HDFS

[jira] [Created] (HDFS-17411) [FGL] Client RPCs involving snapshot support fine-grained lock

2024-03-07 Thread ZanderXu (Jira)
ZanderXu created HDFS-17411: --- Summary: [FGL] Client RPCs involving snapshot support fine-grained lock Key: HDFS-17411 URL: https://issues.apache.org/jira/browse/HDFS-17411 Project: Hadoop HDFS

[jira] [Created] (HDFS-17410) [FGL] Client RPCs that changes file attributes supports fine-grained lock

2024-03-07 Thread ZanderXu (Jira)
ZanderXu created HDFS-17410: --- Summary: [FGL] Client RPCs that changes file attributes supports fine-grained lock Key: HDFS-17410 URL: https://issues.apache.org/jira/browse/HDFS-17410 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-17395) [FGL] Use FSLock to protect ErasureCodingPolicy related operations

2024-03-06 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17395. - Hadoop Flags: Reviewed Resolution: Fixed > [FGL] Use FSLock to protect ErasureCodingPolicy rela

[jira] [Resolved] (HDFS-17406) Suppress UnresolvedPathException in hdfs router log

2024-03-01 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17406. - Fix Version/s: 3.5.0 Hadoop Flags: Reviewed Resolution: Fixed > Suppr

[jira] [Created] (HDFS-17405) [FGL] Using different metric name to trace FGL and Global lock

2024-02-28 Thread ZanderXu (Jira)
ZanderXu created HDFS-17405: --- Summary: [FGL] Using different metric name to trace FGL and Global lock Key: HDFS-17405 URL: https://issues.apache.org/jira/browse/HDFS-17405 Project: Hadoop HDFS

[jira] [Created] (HDFS-17398) [FGL] Implement the FGL lock

2024-02-26 Thread ZanderXu (Jira)
ZanderXu created HDFS-17398: --- Summary: [FGL] Implement the FGL lock Key: HDFS-17398 URL: https://issues.apache.org/jira/browse/HDFS-17398 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-17395) [FGL] Use FSLock to protect ErasureCodingPolicy related operations

2024-02-21 Thread ZanderXu (Jira)
ZanderXu created HDFS-17395: --- Summary: [FGL] Use FSLock to protect ErasureCodingPolicy related operations Key: HDFS-17395 URL: https://issues.apache.org/jira/browse/HDFS-17395 Project: Hadoop HDFS

[jira] [Created] (HDFS-17394) [FGL] Remove unused WriteHoldCount of FSNamesystemLock

2024-02-21 Thread ZanderXu (Jira)
ZanderXu created HDFS-17394: --- Summary: [FGL] Remove unused WriteHoldCount of FSNamesystemLock Key: HDFS-17394 URL: https://issues.apache.org/jira/browse/HDFS-17394 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-17393) Remove unused cond in FSNamesystem

2024-02-21 Thread ZanderXu (Jira)
ZanderXu created HDFS-17393: --- Summary: Remove unused cond in FSNamesystem Key: HDFS-17393 URL: https://issues.apache.org/jira/browse/HDFS-17393 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-17390) [FGL] FSDirectory supports this fine-grained locking

2024-02-20 Thread ZanderXu (Jira)
ZanderXu created HDFS-17390: --- Summary: [FGL] FSDirectory supports this fine-grained locking Key: HDFS-17390 URL: https://issues.apache.org/jira/browse/HDFS-17390 Project: Hadoop HDFS Issue Type

Re: Discussion about NameNode Fine-grained locking

2024-02-20 Thread ZanderXu
Thank you everyone for reviewing this ticket. I think if there are no problems with the goal and the overall solution, we are ready to push this ticket forward and I will create some detailed sub-tasks for this ticket. I will split this project into three milestones to make this project cleaner f

[jira] [Created] (HDFS-17389) [FGL] Create RPC minimizes the scope of the global BM lock

2024-02-19 Thread ZanderXu (Jira)
ZanderXu created HDFS-17389: --- Summary: [FGL] Create RPC minimizes the scope of the global BM lock Key: HDFS-17389 URL: https://issues.apache.org/jira/browse/HDFS-17389 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-17388) [FGL] Create RPC supports this fine-grained locking I

2024-02-19 Thread ZanderXu (Jira)
ZanderXu created HDFS-17388: --- Summary: [FGL] Create RPC supports this fine-grained locking I Key: HDFS-17388 URL: https://issues.apache.org/jira/browse/HDFS-17388 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-17387) [FGL] Abstract selectable locking mode

2024-02-19 Thread ZanderXu (Jira)
ZanderXu created HDFS-17387: --- Summary: [FGL] Abstract selectable locking mode Key: HDFS-17387 URL: https://issues.apache.org/jira/browse/HDFS-17387 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-17386) [FGL] Replace the global BM lock with directory tree-based fine-grained lock

2024-02-18 Thread ZanderXu (Jira)
ZanderXu created HDFS-17386: --- Summary: [FGL] Replace the global BM lock with directory tree-based fine-grained lock Key: HDFS-17386 URL: https://issues.apache.org/jira/browse/HDFS-17386 Project: Hadoop

[jira] [Created] (HDFS-17385) [FGL] Replace the global FS locking with directory tree-based fine-grained locking

2024-02-18 Thread ZanderXu (Jira)
ZanderXu created HDFS-17385: --- Summary: [FGL] Replace the global FS locking with directory tree-based fine-grained locking Key: HDFS-17385 URL: https://issues.apache.org/jira/browse/HDFS-17385 Project

[jira] [Created] (HDFS-17384) [FGL] Replace the global lock with global FS Lock and global BM lock

2024-02-18 Thread ZanderXu (Jira)
ZanderXu created HDFS-17384: --- Summary: [FGL] Replace the global lock with global FS Lock and global BM lock Key: HDFS-17384 URL: https://issues.apache.org/jira/browse/HDFS-17384 Project: Hadoop HDFS

[jira] [Created] (HDFS-17366) NameNode Fine-Grained Locking via Namespace Tree

2024-01-31 Thread ZanderXu (Jira)
ZanderXu created HDFS-17366: --- Summary: NameNode Fine-Grained Locking via Namespace Tree Key: HDFS-17366 URL: https://issues.apache.org/jira/browse/HDFS-17366 Project: Hadoop HDFS Issue Type

[jira] [Resolved] (HDFS-17177) ErasureCodingWork reconstruct ignore the block length is Long.MAX_VALUE.

2023-09-10 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17177. - Fix Version/s: 3.4.0 Resolution: Fixed > ErasureCodingWork reconstruct ignore the block length

[jira] [Resolved] (HDFS-17019) Optimize the logic for reconfigure slow peer enable for Namenode

2023-06-07 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-17019. - Fix Version/s: 3.4.0 Resolution: Fixed > Optimize the logic for reconfigure slow peer enable

[jira] [Created] (HDFS-16987) HA Failover may cause some corrupted blocks

2023-04-23 Thread ZanderXu (Jira)
ZanderXu created HDFS-16987: --- Summary: HA Failover may cause some corrupted blocks Key: HDFS-16987 URL: https://issues.apache.org/jira/browse/HDFS-16987 Project: Hadoop HDFS Issue Type: Bug

[jira] [Resolved] (HDFS-16962) The blockReport RPC should not update the lastBlockReportTime if this blockReport is ignored

2023-03-22 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16962. - Resolution: Invalid > The blockReport RPC should not update the lastBlockReportTime if t

[jira] [Created] (HDFS-16962) The blockReport RPC should not update the lastBlockReportTime if this blockReport is ignored

2023-03-21 Thread ZanderXu (Jira)
ZanderXu created HDFS-16962: --- Summary: The blockReport RPC should not update the lastBlockReportTime if this blockReport is ignored Key: HDFS-16962 URL: https://issues.apache.org/jira/browse/HDFS-16962

[jira] [Created] (HDFS-16961) The blockReport RPC should throw UnregisteredNodeException when the storedDN is null

2023-03-21 Thread ZanderXu (Jira)
ZanderXu created HDFS-16961: --- Summary: The blockReport RPC should throw UnregisteredNodeException when the storedDN is null Key: HDFS-16961 URL: https://issues.apache.org/jira/browse/HDFS-16961 Project

[jira] [Created] (HDFS-16960) Remove useless getEditLog().logSync() for setTimes(), setPermission() and setQuota() PRC

2023-03-21 Thread ZanderXu (Jira)
ZanderXu created HDFS-16960: --- Summary: Remove useless getEditLog().logSync() for setTimes(), setPermission() and setQuota() PRC Key: HDFS-16960 URL: https://issues.apache.org/jira/browse/HDFS-16960 Project

[jira] [Created] (HDFS-16933) A race in SerialNumberMap will cause wrong ownership

2023-02-24 Thread ZanderXu (Jira)
ZanderXu created HDFS-16933: --- Summary: A race in SerialNumberMap will cause wrong ownership Key: HDFS-16933 URL: https://issues.apache.org/jira/browse/HDFS-16933 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-16923) The getListing RPC will throw NPE if the path does not exist

2023-02-15 Thread ZanderXu (Jira)
ZanderXu created HDFS-16923: --- Summary: The getListing RPC will throw NPE if the path does not exist Key: HDFS-16923 URL: https://issues.apache.org/jira/browse/HDFS-16923 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-16910) Fix incorrectly initializing RandomAccessFile caused flush performance decreased for JN

2023-02-08 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16910. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Fix incorrectly initializ

[jira] [Resolved] (HDFS-16879) EC : Fsck -blockId shows number of redundant internal block replicas for EC Blocks

2023-01-03 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16879. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > EC : Fsck -blockId sh

[jira] [Resolved] (HDFS-16831) [RBF SBN] GetNamenodesForNameserviceId should shuffle Observer NameNodes every time

2022-12-22 Thread ZanderXu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZanderXu resolved HDFS-16831. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > [RBF

  1   2   >