[jira] [Created] (HDDS-12564) Prevent datanode disks from fully occupied

2025-03-11 Thread Sammi Chen (Jira)
Sammi Chen created HDDS-12564: - Summary: Prevent datanode disks from fully occupied Key: HDDS-12564 URL: https://issues.apache.org/jira/browse/HDDS-12564 Project: Apache Ozone Issue Type: Impro

[jira] [Updated] (HDDS-12555) Combine containerMap and replicaMap in ContainerStateMap

2025-03-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12555: -- Labels: pull-request-available (was: ) > Combine containerMap and replicaMap in ContainerState

[jira] [Created] (HDDS-12563) Cache and reuse ContainerID object

2025-03-11 Thread Nandakumar (Jira)
Nandakumar created HDDS-12563: - Summary: Cache and reuse ContainerID object Key: HDDS-12563 URL: https://issues.apache.org/jira/browse/HDDS-12563 Project: Apache Ozone Issue Type: Bug C

[PR] HDDS-12555. Combine containerMap and replicaMap in ContainerStateMap. [ozone]

2025-03-11 Thread via GitHub
szetszwo opened a new pull request, #8057: URL: https://github.com/apache/ozone/pull/8057 ## What changes were proposed in this pull request? ContainerStateMap has two maps, containerMap and replicaMap. Both of them has the same key type ContainerID. They should be combined into one

[jira] [Updated] (HDDS-12437) [DiskBalancer] Estimate the total size pending to move before disk usage becomes even

2025-03-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12437: -- Labels: pull-request-available (was: ) > [DiskBalancer] Estimate the total size pending to mov

[PR] HDDS-12437. [DiskBalancer] Estimate the total size pending to move before disk usage becomes even [ozone]

2025-03-11 Thread via GitHub
Gargi-jais11 opened a new pull request, #8056: URL: https://github.com/apache/ozone/pull/8056 ## What changes were proposed in this pull request? It will be an estimation value, due to there are other activities going on, such as block deletion, new container creation, container replica d

Re: [PR] HDDS-12097. Enhance Container Key Mapper for Faster Processing. [ozone]

2025-03-11 Thread via GitHub
ArafatKhan2198 commented on code in PR #8002: URL: https://github.com/apache/ozone/pull/8002#discussion_r1990576866 ## hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/ReconControllerModule.java: ## @@ -131,7 +130,6 @@ static class ReconOmTaskBindingModule extends

Re: [PR] HDDS-12053. Make print-log-dag command run locally and offline [ozone]

2025-03-11 Thread via GitHub
Tejaskriya commented on code in PR #8016: URL: https://github.com/apache/ozone/pull/8016#discussion_r1990557818 ## hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/compaction/log/PopulateCompactionTable.java: ## @@ -0,0 +1,240 @@ +/* + * Licensed to the Apach

Re: [PR] HDDS-12553. Support ozone admin container list to print list in Json array [ozone]

2025-03-11 Thread via GitHub
kerneltime commented on PR #8050: URL: https://github.com/apache/ozone/pull/8050#issuecomment-2716217836 @errose28 for now I am not breaking the old CLI output, if developers feel strongly we can do it as a separate PR with updates to existing tests that fail. -- This is an automated mess

Re: [PR] HDDS-12488. S3G should handle the signature calculation with trailers [ozone]

2025-03-11 Thread via GitHub
jojochuang commented on code in PR #8020: URL: https://github.com/apache/ozone/pull/8020#discussion_r1988096700 ## hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/s3/awssdk/v2/AbstractS3SDKV2Tests.java: ## @@ -0,0 +1,288 @@ +/* + * Licensed to the Apache Soft

Re: [PR] HDDS-11454. Ozone Ranger Integration in docker [ozone]

2025-03-11 Thread via GitHub
adoroszlai commented on code in PR #8037: URL: https://github.com/apache/ozone/pull/8037#discussion_r1988426189 ## hadoop-ozone/dist/src/main/compose/ozone/.env: ## @@ -19,3 +19,4 @@ HADOOP_IMAGE=apache/hadoop OZONE_RUNNER_VERSION=${docker.ozone-runner.version} OZONE_RUNNER_IM

Re: [PR] HDDS-12432. [DiskBalancer] Respect property "hdds.datanode.disk.balancer.max.disk.throughputInMBPerSec" [ozone]

2025-03-11 Thread via GitHub
Gargi-jais11 commented on code in PR #8006: URL: https://github.com/apache/ozone/pull/8006#discussion_r1986810707 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/diskbalancer/DiskBalancerService.java: ## @@ -363,17 +363,19 @@ public BackgroundTas

[jira] [Created] (HDDS-12489) Intermittent timeout in TestSCMContainerManagerMetrics.testReportProcessingMetrics

2025-03-11 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-12489: --- Summary: Intermittent timeout in TestSCMContainerManagerMetrics.testReportProcessingMetrics Key: HDDS-12489 URL: https://issues.apache.org/jira/browse/HDDS-12489

[jira] [Updated] (HDDS-12562) Reclaimable Directory entry filter for reclaiming deleted directory entries

2025-03-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12562: -- Labels: pull-request-available (was: ) > Reclaimable Directory entry filter for reclaiming del

[PR] HDDS-12562. Reclaimable Directory entry filter for reclaiming deleted directory entries [ozone]

2025-03-11 Thread via GitHub
swamirishi opened a new pull request, #8055: URL: https://github.com/apache/ozone/pull/8055 ## What changes were proposed in this pull request? Currently deleted directory entries are reclaimed by just peeking into the previous snapshot without taking locks and the subdirectories and sub

Re: [PR] HDDS-12553. Support ozone admin container list to print list in Json array [ozone]

2025-03-11 Thread via GitHub
kerneltime commented on PR #8050: URL: https://github.com/apache/ozone/pull/8050#issuecomment-2716087082 @errose28 @adoroszlai thank you for the review, I have addressed the review comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[jira] [Assigned] (HDDS-12551) Replace dnsToUuidMap with dnsToDnIdMap in SCMNodeManager

2025-03-11 Thread Chia-Chuan Yu (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Chuan Yu reassigned HDDS-12551: Assignee: Chia-Chuan Yu > Replace dnsToUuidMap with dnsToDnIdMap in SCMNodeManager > -

Re: [PR] HDDS-12483. Quasi Closed Stuck should have 2 replicas of each origin [ozone]

2025-03-11 Thread via GitHub
siddhantsangwan commented on code in PR #8014: URL: https://github.com/apache/ozone/pull/8014#discussion_r1987378369 ## hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/QuasiClosedStuckUnderReplicationHandler.java: ## @@ -0,0 +1,158 @@ +/* +

[jira] [Created] (HDDS-12562) Reclaimable Directory entry filter for reclaiming deleted directory entries

2025-03-11 Thread Swaminathan Balachandran (Jira)
Swaminathan Balachandran created HDDS-12562: --- Summary: Reclaimable Directory entry filter for reclaiming deleted directory entries Key: HDDS-12562 URL: https://issues.apache.org/jira/browse/HDDS-12562

[jira] [Updated] (HDDS-12561) Reclaimable Rename entry filter for reclaiming renaming entries

2025-03-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12561: -- Labels: pull-request-available (was: ) > Reclaimable Rename entry filter for reclaiming renami

[PR] HDDS-12561. Reclaimable Rename entry filter for reclaiming renaming entries [ozone]

2025-03-11 Thread via GitHub
swamirishi opened a new pull request, #8054: URL: https://github.com/apache/ozone/pull/8054 ## What changes were proposed in this pull request? Currently rename entries are never reclaimed and these entries can be reclaimed on snapshot deep cleaning by just peeking into the previous snaps

[jira] [Created] (HDDS-12561) Reclaimable Rename entry filter for reclaiming renaming entries

2025-03-11 Thread Swaminathan Balachandran (Jira)
Swaminathan Balachandran created HDDS-12561: --- Summary: Reclaimable Rename entry filter for reclaiming renaming entries Key: HDDS-12561 URL: https://issues.apache.org/jira/browse/HDDS-12561 P

Re: [PR] HDDS-11603. Reclaimable Filter for Snapshots garbage reclaimation [ozone]

2025-03-11 Thread via GitHub
swamirishi commented on code in PR #7345: URL: https://github.com/apache/ozone/pull/7345#discussion_r1983758730 ## hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/snapshot/filter/ReclaimableKeyFilter.java: ## @@ -0,0 +1,275 @@ +/* + * Licensed to the Apache S

[jira] [Updated] (HDDS-12560) Reclaimable Filter for Snaphost Garbage Collections

2025-03-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12560: -- Labels: pull-request-available (was: ) > Reclaimable Filter for Snaphost Garbage Collections >

[PR] HDDS-12560. Reclaimable Filter for Snaphost Garbage Collections [ozone]

2025-03-11 Thread via GitHub
swamirishi opened a new pull request, #8053: URL: https://github.com/apache/ozone/pull/8053 ## What changes were proposed in this pull request? Currently all garbage collection threads peek into previous snapshots to check whether a particular deleted key/deleted Directory/ RenameEntry ca

[jira] [Updated] (HDDS-12560) Reclaimable Filter for Snaphost Garbage Collections

2025-03-11 Thread Swaminathan Balachandran (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swaminathan Balachandran updated HDDS-12560: Description: Currently all garbage collection threads peek into previous sn

[jira] [Created] (HDDS-12560) Reclaimable Filter for Snaphost Garbage Collections

2025-03-11 Thread Swaminathan Balachandran (Jira)
Swaminathan Balachandran created HDDS-12560: --- Summary: Reclaimable Filter for Snaphost Garbage Collections Key: HDDS-12560 URL: https://issues.apache.org/jira/browse/HDDS-12560 Project: Apach

[jira] [Updated] (HDDS-12559) Implement Bulk Ozone Locks for taking locks on multiple snapshots

2025-03-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12559: -- Labels: pull-request-available (was: ) > Implement Bulk Ozone Locks for taking locks on multip

[PR] HDDS-12559. Implement Bulk Ozone Locks for taking locks on multiple snapshots [ozone]

2025-03-11 Thread via GitHub
swamirishi opened a new pull request, #8052: URL: https://github.com/apache/ozone/pull/8052 ## What changes were proposed in this pull request? Currently we don't have the ability to take OM locks on a set of keys. We need this functionality to take locks on multiple snapshots at once whi

[jira] [Updated] (HDDS-12559) Implement Bulk Ozone Locks for taking locks on multiple snapshots

2025-03-11 Thread Swaminathan Balachandran (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swaminathan Balachandran updated HDDS-12559: Description: Currently we don't have the ability to take OM locks on a set

[jira] [Created] (HDDS-12559) Implement Bulk Ozone Locks for taking locks on multiple snapshots

2025-03-11 Thread Swaminathan Balachandran (Jira)
Swaminathan Balachandran created HDDS-12559: --- Summary: Implement Bulk Ozone Locks for taking locks on multiple snapshots Key: HDDS-12559 URL: https://issues.apache.org/jira/browse/HDDS-12559

[jira] [Updated] (HDDS-11603) Reclaimable Filter for Snapshots garbage reclaimation

2025-03-11 Thread Swaminathan Balachandran (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swaminathan Balachandran updated HDDS-11603: Parent: HDDS-12558 Issue Type: Sub-task (was: Bug) > Reclaimable F

[jira] [Updated] (HDDS-12558) Snapshot Garbage Collection Issues

2025-03-11 Thread Swaminathan Balachandran (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swaminathan Balachandran updated HDDS-12558: Description: This is a parent Jira targetting to resolve all Snapshot Garba

[jira] [Created] (HDDS-12558) Snapshot Garbage Collection Issues

2025-03-11 Thread Swaminathan Balachandran (Jira)
Swaminathan Balachandran created HDDS-12558: --- Summary: Snapshot Garbage Collection Issues Key: HDDS-12558 URL: https://issues.apache.org/jira/browse/HDDS-12558 Project: Apache Ozone

[jira] [Assigned] (HDDS-12558) Snapshot Garbage Collection Issues

2025-03-11 Thread Swaminathan Balachandran (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swaminathan Balachandran reassigned HDDS-12558: --- Assignee: Swaminathan Balachandran > Snapshot Garbage Collection Iss

[jira] [Updated] (HDDS-11603) Reclaimable Filter for Snapshots garbage reclaimation

2025-03-11 Thread Swaminathan Balachandran (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swaminathan Balachandran updated HDDS-11603: Parent: (was: HDDS-8544) Issue Type: Bug (was: Sub-task) > Rec

[jira] [Updated] (HDDS-11603) Reclaimable Filter for Snapshots garbage reclaimation

2025-03-11 Thread Swaminathan Balachandran (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swaminathan Balachandran updated HDDS-11603: Description: We have implemented the key reference check for snapshots in m

Re: [PR] HDDS-11603. Reclaimable Filter for Snapshots garbage reclaimation [ozone]

2025-03-11 Thread via GitHub
swamirishi commented on code in PR #7345: URL: https://github.com/apache/ozone/pull/7345#discussion_r1983754465 ## hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/snapshot/filter/ReclaimableFilter.java: ## @@ -0,0 +1,220 @@ +/* + * Licensed to the Apache Soft

[jira] [Resolved] (HDDS-12322) EC read and reconstruct fail when datanode in maintenance

2025-03-11 Thread Sammi Chen (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sammi Chen resolved HDDS-12322. --- Resolution: Not A Problem > EC read and reconstruct fail when datanode in maintenance > -

[jira] [Updated] (HDDS-12557) Snapshot tarball progress indicator

2025-03-11 Thread Saketa Chalamchala (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saketa Chalamchala updated HDDS-12557: -- Parent: HDDS-8544 Issue Type: Sub-task (was: Task) > Snapshot tarball progress

[jira] [Updated] (HDDS-12557) Snapshot tarball progress indicator

2025-03-11 Thread Saketa Chalamchala (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saketa Chalamchala updated HDDS-12557: -- Parent: (was: HDDS-6517) Issue Type: Task (was: Sub-task) > Snapshot tarba

[jira] [Created] (HDDS-12557) Snapshot tarball progress indicator

2025-03-11 Thread Saketa Chalamchala (Jira)
Saketa Chalamchala created HDDS-12557: - Summary: Snapshot tarball progress indicator Key: HDDS-12557 URL: https://issues.apache.org/jira/browse/HDDS-12557 Project: Apache Ozone Issue Type

Re: [PR] HDDS-11603. Reclaimable Filter for Snapshots garbage reclaimation [ozone]

2025-03-11 Thread via GitHub
swamirishi commented on code in PR #7345: URL: https://github.com/apache/ozone/pull/7345#discussion_r1983756881 ## hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/snapshot/filter/ReclaimableFilter.java: ## @@ -0,0 +1,220 @@ +/* + * Licensed to the Apache Soft

[PR] HDDS-12552. Fix Raw use of parameterized class 'SCMCommand'. [ozone]

2025-03-11 Thread via GitHub
nandakumar131 opened a new pull request, #8048: URL: https://github.com/apache/ozone/pull/8048 ## What changes were proposed in this pull request? Fix Raw use of parameterized class 'SCMCommand'. ## What is the link to the Apache JIRA HDDS-12552 ## How was this pat

Re: [PR] HDDS-12488. S3G should handle the signature calculation with trailers [ozone]

2025-03-11 Thread via GitHub
ivandika3 commented on code in PR #8020: URL: https://github.com/apache/ozone/pull/8020#discussion_r1988227714 ## hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/SignedChunksInputStream.java: ## @@ -23,19 +23,26 @@ import java.util.regex.Pattern; /** - * Inpu

[jira] [Updated] (HDDS-11454) Ranger integration for Docker dev environment

2025-03-11 Thread Abhishek Kumar (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Kumar updated HDDS-11454: -- Attachment: ozone-om-1.log > Ranger integration for Docker dev environment > --

Re: [PR] HDDS-12432. [DiskBalancer] Respect property "hdds.datanode.disk.balancer.max.disk.throughputInMBPerSec" [ozone]

2025-03-11 Thread via GitHub
ChenSammi merged PR #8006: URL: https://github.com/apache/ozone/pull/8006 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.apach

[jira] [Commented] (HDDS-12556) Use closable handle to allocate and release committed bytes

2025-03-11 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17934327#comment-17934327 ] Ethan Rose commented on HDDS-12556: --- One possible implementation is that {{VolumeChoosi

Re: [PR] HDDS-12235. Reserve space on DN during container import operation. [ozone]

2025-03-11 Thread via GitHub
errose28 commented on code in PR #7981: URL: https://github.com/apache/ozone/pull/7981#discussion_r1990018252 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/DownloadAndImportReplicator.java: ## @@ -70,9 +77,19 @@ public void replicat

[jira] [Created] (HDDS-12556) Use closable handle to allocate and release committed bytes

2025-03-11 Thread Ethan Rose (Jira)
Ethan Rose created HDDS-12556: - Summary: Use closable handle to allocate and release committed bytes Key: HDDS-12556 URL: https://issues.apache.org/jira/browse/HDDS-12556 Project: Apache Ozone I

Re: [PR] HDDS-12233. Atomically import a container [ozone]

2025-03-11 Thread via GitHub
ptlrs commented on PR #7934: URL: https://github.com/apache/ozone/pull/7934#issuecomment-2715504556 Hi @umamaheswararao, @swamirishi, @ChenSammi, @errose28, @kerneltime could you please review this PR. Thanks. -- This is an automated message from the Apache Git Service. To respond to the

[jira] [Resolved] (HDDS-12419) Space tracking for Imported Containers in HDDS Volumes

2025-03-11 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Rose resolved HDDS-12419. --- Resolution: Duplicate > Space tracking for Imported Containers in HDDS Volumes >

Re: [PR] HDDS-12432. [DiskBalancer] Respect property "hdds.datanode.disk.balancer.max.disk.throughputInMBPerSec" [ozone]

2025-03-11 Thread via GitHub
ChenSammi commented on code in PR #8006: URL: https://github.com/apache/ozone/pull/8006#discussion_r1986779463 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/diskbalancer/DiskBalancerService.java: ## @@ -363,17 +363,19 @@ public BackgroundTaskQu

Re: [PR] HDDS-12053. Make print-log-dag command run locally and offline [ozone]

2025-03-11 Thread via GitHub
Tejaskriya commented on PR #8016: URL: https://github.com/apache/ozone/pull/8016#issuecomment-2705807167 @adoroszlai thank you for the review! I have addressed all of the comments -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[jira] [Comment Edited] (HDDS-12524) Reuse TestDataUtil.createKey in more tests

2025-03-11 Thread Chia-Chuan Yu (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17933857#comment-17933857 ] Chia-Chuan Yu edited comment on HDDS-12524 at 3/10/25 12:24 PM: ---

Re: [PR] HDDS-12057. Implement command ozone debug replicas verify checksums [ozone]

2025-03-11 Thread via GitHub
ptlrs commented on code in PR #7748: URL: https://github.com/apache/ozone/pull/7748#discussion_r1989906510 ## hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/replicas/ReplicasVerify.java: ## @@ -0,0 +1,134 @@ +/* + * Licensed to the Apache Software Foundation (ASF

Re: [PR] HDDS-12235. Reserve space on DN during container import operation. [ozone]

2025-03-11 Thread via GitHub
errose28 commented on code in PR #7981: URL: https://github.com/apache/ozone/pull/7981#discussion_r1989895457 ## hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/replication/TestReplicationSupervisor.java: ## @@ -327,6 +347,108 @@ public void testDo

Re: [PR] HDDS-11576. Create a separate S3 client factory [ozone]

2025-03-11 Thread via GitHub
adoroszlai commented on code in PR #8051: URL: https://github.com/apache/ozone/pull/8051#discussion_r1989887710 ## hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/s3/awssdk/v1/AbstractS3SDKV1Tests.java: ## @@ -186,7 +186,7 @@ static void startCluster(OzoneCon

Re: [PR] HDDS-12235. Reserve space on DN during container import operation. [ozone]

2025-03-11 Thread via GitHub
errose28 commented on code in PR #7981: URL: https://github.com/apache/ozone/pull/7981#discussion_r1989891509 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/DownloadAndImportReplicator.java: ## @@ -70,9 +77,19 @@ public void replicat

[jira] [Updated] (HDDS-12538) Update README/CONTRIBUTING in ozone-docker

2025-03-11 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-12538: Status: Patch Available (was: In Progress) > Update README/CONTRIBUTING in ozone-docker >

Re: [PR] HDDS-12483. Quasi Closed Stuck should have 2 replicas of each origin [ozone]

2025-03-11 Thread via GitHub
sodonnel commented on PR #8014: URL: https://github.com/apache/ozone/pull/8014#issuecomment-2711150955 > uppose there are two unique origins, and this new logic has made it such that there are two replicas of each origin. So in total there are 4 replicas. Will `QuasiClosedStuckReplicationCh

Re: [PR] HDDS-12057. Implement command ozone debug replicas verify checksums [ozone]

2025-03-11 Thread via GitHub
ptlrs commented on code in PR #7748: URL: https://github.com/apache/ozone/pull/7748#discussion_r1989872625 ## hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/replicas/ReplicasVerify.java: ## @@ -0,0 +1,134 @@ +/* + * Licensed to the Apache Software Foundation (ASF

Re: [PR] HDDS-12057. Implement command ozone debug replicas verify checksums [ozone]

2025-03-11 Thread via GitHub
ptlrs commented on code in PR #7748: URL: https://github.com/apache/ozone/pull/7748#discussion_r1989871667 ## hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/replicas/ReplicasVerify.java: ## @@ -0,0 +1,134 @@ +/* + * Licensed to the Apache Software Foundation (ASF

Re: [PR] HDDS-12057. Implement command ozone debug replicas verify checksums [ozone]

2025-03-11 Thread via GitHub
ptlrs commented on code in PR #7748: URL: https://github.com/apache/ozone/pull/7748#discussion_r1989866930 ## hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/replicas/ReplicasVerify.java: ## @@ -0,0 +1,134 @@ +/* + * Licensed to the Apache Software Foundation (ASF

Re: [PR] HDDS-12057. Implement command ozone debug replicas verify checksums [ozone]

2025-03-11 Thread via GitHub
ptlrs commented on code in PR #7748: URL: https://github.com/apache/ozone/pull/7748#discussion_r1989866256 ## hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/replicas/KeyParts.java: ## @@ -0,0 +1,42 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[jira] [Updated] (HDDS-12383) Fix spotbugs warnings in prod

2025-03-11 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-12383: Status: Patch Available (was: Open) > Fix spotbugs warnings in prod >

[jira] [Updated] (HDDS-11576) Create a separate S3 client factory

2025-03-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-11576: -- Labels: pull-request-available (was: ) > Create a separate S3 client factory > --

[PR] HDDS-11576. Create a separate S3 client factory [ozone]

2025-03-11 Thread via GitHub
peterxcli opened a new pull request, #8051: URL: https://github.com/apache/ozone/pull/8051 ## What changes were proposed in this pull request? Introduce a separate factory class for creating the client, and remove AmazonS3 from MiniOzoneCluster interface. Related comment: http

Re: [PR] HDDS-12553. Support ozone admin container list to print list in Json array [ozone]

2025-03-11 Thread via GitHub
adoroszlai commented on code in PR #8050: URL: https://github.com/apache/ozone/pull/8050#discussion_r1989826869 ## hadoop-ozone/dist/src/main/smoketest/admincli/container.robot: ## @@ -96,6 +96,33 @@ List all containers from a particular container ID ${output} = Exe

Re: [PR] HDDS-12553. Support ozone admin container list to print list in Json array [ozone]

2025-03-11 Thread via GitHub
kerneltime commented on code in PR #8050: URL: https://github.com/apache/ozone/pull/8050#discussion_r1989820692 ## hadoop-ozone/dist/src/main/smoketest/admincli/container.robot: ## @@ -96,6 +96,33 @@ List all containers from a particular container ID ${output} = Exe

[jira] [Created] (HDDS-12555) Combine containerMap and replicaMap in ContainerStateMap

2025-03-11 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-12555: - Summary: Combine containerMap and replicaMap in ContainerStateMap Key: HDDS-12555 URL: https://issues.apache.org/jira/browse/HDDS-12555 Project: Apache Ozone Issue

Re: [PR] HDDS-12553. Support ozone admin container list to print list in Json array [ozone]

2025-03-11 Thread via GitHub
kerneltime commented on code in PR #8050: URL: https://github.com/apache/ozone/pull/8050#discussion_r1989819489 ## hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/container/ListSubcommand.java: ## @@ -76,6 +76,11 @@ public class ListSubcommand extends ScmSubcomman

Re: [PR] HDDS-12553. Support ozone admin container list to print list in Json array [ozone]

2025-03-11 Thread via GitHub
kerneltime commented on code in PR #8050: URL: https://github.com/apache/ozone/pull/8050#discussion_r1989816086 ## hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/container/ListSubcommand.java: ## @@ -90,11 +95,35 @@ public class ListSubcommand extends ScmSubcomma

Re: [PR] HDDS-11454. Ozone Ranger Integration in docker [ozone]

2025-03-11 Thread via GitHub
adoroszlai commented on code in PR #8037: URL: https://github.com/apache/ozone/pull/8037#discussion_r1987915474 ## hadoop-ozone/dist/src/main/compose/ozone/docker-compose.ranger.yaml: ## @@ -0,0 +1,92 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more co

Re: [PR] HDDS-11763. Implement container repair logic within datanodes. [ozone]

2025-03-11 Thread via GitHub
aswinshakil commented on code in PR #7474: URL: https://github.com/apache/ozone/pull/7474#discussion_r1987961216 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/BlockManagerImpl.java: ## @@ -121,7 +121,7 @@ public long persistPutBlo

[jira] [Created] (HDDS-12535) Intermittent failure in TestContainerReportHandling

2025-03-11 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-12535: --- Summary: Intermittent failure in TestContainerReportHandling Key: HDDS-12535 URL: https://issues.apache.org/jira/browse/HDDS-12535 Project: Apache Ozone

[jira] [Resolved] (HDDS-9040) Enhance debug tool to create a visual graph of compaction log DAG

2025-03-11 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-9040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Rose resolved HDDS-9040. -- Resolution: Duplicate > Enhance debug tool to create a visual graph of compaction log DAG >

[jira] [Commented] (HDDS-12053) Make `ozone debug om compaction-dag` run locally and offline

2025-03-11 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17933993#comment-17933993 ] Ethan Rose commented on HDDS-12053: --- Just resolved it in favor of this task. > Make `o

Re: [PR] HDDS-12553. Support ozone admin container list to print list in Json array [ozone]

2025-03-11 Thread via GitHub
errose28 commented on code in PR #8050: URL: https://github.com/apache/ozone/pull/8050#discussion_r1989770192 ## hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/container/ListSubcommand.java: ## @@ -90,11 +95,35 @@ public class ListSubcommand extends ScmSubcommand

Re: [PR] HDDS-12488. S3G should handle the signature calculation with trailers [ozone]

2025-03-11 Thread via GitHub
ivandika3 commented on code in PR #8020: URL: https://github.com/apache/ozone/pull/8020#discussion_r1988237967 ## hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/s3/awssdk/v2/AbstractS3SDKV2Tests.java: ## @@ -0,0 +1,288 @@ +/* + * Licensed to the Apache Softw

Re: [PR] HDDS-12057. Implement command ozone debug replicas verify checksums [ozone]

2025-03-11 Thread via GitHub
sarvekshayr commented on code in PR #7748: URL: https://github.com/apache/ozone/pull/7748#discussion_r1988714561 ## hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/replicas/ReplicasVerify.java: ## @@ -0,0 +1,134 @@ +/* + * Licensed to the Apache Software Foundatio

[jira] [Updated] (HDDS-12537) Selective checks: skip tests for PMD ruleset change

2025-03-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12537: -- Labels: pull-request-available (was: ) > Selective checks: skip tests for PMD ruleset change >

Re: [PR] HDDS-12534. Remove drop_column_family command from ozone debug ldb [ozone]

2025-03-11 Thread via GitHub
adoroszlai commented on PR #8038: URL: https://github.com/apache/ozone/pull/8038#issuecomment-2709973206 > How about having this option in repair ? I think if we need to drop a specific column family due to a specific incompatibility that may be reported in the future, we can add a re

Re: [PR] HDDS-12519. Generate auto-complete script for Ozone commands. [ozone]

2025-03-11 Thread via GitHub
nandakumar131 commented on code in PR #8030: URL: https://github.com/apache/ozone/pull/8030#discussion_r1987570973 ## hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/utils/AutoCompletion.java: ## @@ -0,0 +1,164 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] HDDS-12543. Remove duplicate license information [ozone]

2025-03-11 Thread via GitHub
ptlrs commented on code in PR #8045: URL: https://github.com/apache/ozone/pull/8045#discussion_r1988315824 ## hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/TestContainerStateMachine.java: ## @@ -61,22 +61,6 @@ import

[jira] [Created] (HDDS-12496) Use TextFormat#shortDebugString to flatten proto message in SCMDatanodeProtocolServer

2025-03-11 Thread Nandakumar (Jira)
Nandakumar created HDDS-12496: - Summary: Use TextFormat#shortDebugString to flatten proto message in SCMDatanodeProtocolServer Key: HDDS-12496 URL: https://issues.apache.org/jira/browse/HDDS-12496 Project

Re: [PR] HDDS-11763. Implement container repair logic within datanodes. [ozone]

2025-03-11 Thread via GitHub
aswinshakil commented on code in PR #7474: URL: https://github.com/apache/ozone/pull/7474#discussion_r1987963551 ## hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueHandler.java: ## @@ -1483,21 +1483,152 @@ public void deleteContaine

Re: [PR] HDDS-12534. Remove drop_column_family command from ozone debug ldb [ozone]

2025-03-11 Thread via GitHub
adoroszlai commented on PR #8038: URL: https://github.com/apache/ozone/pull/8038#issuecomment-2712744693 Thanks @Tejaskriya for the patch, @errose28, @nandakumar131, @sumitagrawl for the review. -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] HDDS-12435. [DiskBalancer] Add success move count and fail move count in status report [ozone]

2025-03-11 Thread via GitHub
ChenSammi commented on code in PR #8009: URL: https://github.com/apache/ozone/pull/8009#discussion_r1988329107 ## hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/DiskBalancerManager.java: ## @@ -260,11 +260,20 @@ private HddsProtos.DatanodeDiskBalancerInfoPr

Re: [PR] HDDS-12421. ContainerReportHandler should not make the call to delete replicas [ozone]

2025-03-11 Thread via GitHub
swamirishi commented on code in PR #7976: URL: https://github.com/apache/ozone/pull/7976#discussion_r1984157418 ## hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/replication/health/TestDeletingContainerHandler.java: ## @@ -217,7 +217,7 @@ public void t

Re: [PR] HDDS-12435. [DiskBalancer] Add success move count and fail move count in status report [ozone]

2025-03-11 Thread via GitHub
ChenSammi commented on code in PR #8009: URL: https://github.com/apache/ozone/pull/8009#discussion_r1986722077 ## hadoop-hdds/interface-server/src/main/proto/ScmServerDatanodeHeartbeatProtocol.proto: ## @@ -499,6 +499,8 @@ message DiskBalancerReportProto { required bool isRun

Re: [PR] HDDS-11603. Reclaimable Filter for Snapshots garbage reclaimation [ozone]

2025-03-11 Thread via GitHub
swamirishi commented on code in PR #7345: URL: https://github.com/apache/ozone/pull/7345#discussion_r1983757513 ## hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/snapshot/filter/TestReclaimableFilter.java: ## @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache S

Re: [PR] HDDS-11603. Reclaimable Filter for Snapshots garbage reclaimation [ozone]

2025-03-11 Thread via GitHub
swamirishi commented on code in PR #7345: URL: https://github.com/apache/ozone/pull/7345#discussion_r1983753099 ## hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/snapshot/filter/ReclaimableFilter.java: ## @@ -0,0 +1,220 @@ +/* + * Licensed to the Apache Soft

Re: [PR] HDDS-12004. [Doc] Other applications known to work with Ozone [ozone]

2025-03-11 Thread via GitHub
jojochuang commented on code in PR #8042: URL: https://github.com/apache/ozone/pull/8042#discussion_r1988113730 ## hadoop-hdds/docs/content/integration/ThirdParties.md: ## @@ -0,0 +1,63 @@ +--- +title: Third-Party Integrations +weight: 4 +menu: + main: + parent: "Applicat

Re: [PR] HDDS-12053. Make print-log-dag command run locally and offline [ozone]

2025-03-11 Thread via GitHub
errose28 commented on code in PR #8016: URL: https://github.com/apache/ozone/pull/8016#discussion_r1987674194 ## hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/om/CompactionLogDagPrinter.java: ## @@ -0,0 +1,202 @@ +/* + * Licensed to the Apache Software Foundatio

[jira] [Created] (HDDS-12534) Remove drop_column_family command from ozone debug ldb

2025-03-11 Thread Tejaskriya Madhan (Jira)
Tejaskriya Madhan created HDDS-12534: Summary: Remove drop_column_family command from ozone debug ldb Key: HDDS-12534 URL: https://issues.apache.org/jira/browse/HDDS-12534 Project: Apache Ozone

[jira] [Updated] (HDDS-12538) Update README/CONTRIBUTING in ozone-docker

2025-03-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12538: -- Labels: pull-request-available (was: ) > Update README/CONTRIBUTING in ozone-docker >

Re: [PR] HDDS-12483. Quasi Closed Stuck should have 2 replicas of each origin [ozone]

2025-03-11 Thread via GitHub
sodonnel commented on code in PR #8014: URL: https://github.com/apache/ozone/pull/8014#discussion_r1989115240 ## hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/QuasiClosedStuckUnderReplicationHandler.java: ## @@ -0,0 +1,158 @@ +/* + * Licen

Re: [PR] HDDS-12518. Compact RocksDB after consecutive deletions [ozone]

2025-03-11 Thread via GitHub
errose28 commented on code in PR #8029: URL: https://github.com/apache/ozone/pull/8029#discussion_r1987679150 ## hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/service/CompactionService.java: ## @@ -0,0 +1,164 @@ +/* + * Licensed to the Apache Software Found

[jira] [Assigned] (HDDS-11817) Limit the number of snapshots

2025-03-11 Thread Saketa Chalamchala (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saketa Chalamchala reassigned HDDS-11817: - Assignee: Saketa Chalamchala > Limit the number of snapshots > -

  1   2   >