Re: [PR] HDDS-9377. Implement some metrics with RW-lock instead of synchronization [ozone]

2025-02-03 Thread via GitHub
juncevich commented on PR #7708: URL: https://github.com/apache/ozone/pull/7708#issuecomment-2633132555 > Thanks @juncevich for the patch. It seems this removes synchronization in some individual metrics, but not in `MetricsSytemImpl` itself. (see also [HDDS-12193](https://issues.apache.org

Re: [PR] HDDS-12159. Remove redundant seek for rocksDBs [ozone]

2025-02-03 Thread via GitHub
sadanand48 commented on code in PR #7794: URL: https://github.com/apache/ozone/pull/7794#discussion_r1940655895 ## hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java: ## @@ -2142,9 +2139,7 @@ public DeleteKeysResult getPendingDeletionSubFiles

Re: [PR] HDDS-12192. Fix TestOzoneShellHA and extract set-bucket-encryption test case [ozone]

2025-02-03 Thread via GitHub
adoroszlai commented on code in PR #7802: URL: https://github.com/apache/ozone/pull/7802#discussion_r1940649596 ## hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneShellHA.java: ## @@ -2190,8 +2195,7 @@ public void testClientBucketLayoutValidati

Re: [PR] HDDS-12192. Fix TestOzoneShellHA and extract set-bucket-encryption test case [ozone]

2025-02-03 Thread via GitHub
adoroszlai commented on code in PR #7802: URL: https://github.com/apache/ozone/pull/7802#discussion_r1940649596 ## hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneShellHA.java: ## @@ -2190,8 +2195,7 @@ public void testClientBucketLayoutValidati

Re: [PR] HDDS-12192. Fix TestOzoneShellHA and extract set-bucket-encryption test case [ozone]

2025-02-03 Thread via GitHub
sadanand48 commented on code in PR #7802: URL: https://github.com/apache/ozone/pull/7802#discussion_r1940645288 ## hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneShellHA.java: ## @@ -2190,8 +2195,7 @@ public void testClientBucketLayoutValidati

Re: [PR] HDDS-12159. Remove redundant seek for rocksDBs [ozone]

2025-02-03 Thread via GitHub
nandakumar131 commented on PR #7794: URL: https://github.com/apache/ozone/pull/7794#issuecomment-2633092773 > Also isn't the iterator invalid without calling any of the seek methods, I believe this is the reason we have called seek() api's after initialising iterators . Has Rocksdb behaviou

Re: [PR] HDDS-12159. Remove redundant seek for rocksDBs [ozone]

2025-02-03 Thread via GitHub
nandakumar131 commented on code in PR #7794: URL: https://github.com/apache/ozone/pull/7794#discussion_r1940633548 ## hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java: ## @@ -2142,9 +2139,7 @@ public DeleteKeysResult getPendingDeletionSubFi

Re: [PR] HDDS-12159. Remove redundant seek for rocksDBs [ozone]

2025-02-03 Thread via GitHub
Tejaskriya commented on code in PR #7794: URL: https://github.com/apache/ozone/pull/7794#discussion_r1940633001 ## hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java: ## @@ -2142,9 +2139,7 @@ public DeleteKeysResult getPendingDeletionSubFiles

[jira] [Commented] (HDDS-12087) TransactionToDNeCommitMap too large causes GC to pause for a long time

2025-02-03 Thread Aryan Gupta (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923600#comment-17923600 ] Aryan Gupta commented on HDDS-12087: Hi [~Nicholas Niu], Are you planning to work on

[jira] [Updated] (HDDS-12178) Expand transitive test dependencies

2025-02-03 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-12178: Status: Patch Available (was: In Progress) > Expand transitive test dependencies > ---

[jira] [Updated] (HDDS-12192) Split and fix TestOzoneShellHA

2025-02-03 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-12192: Status: Patch Available (was: In Progress) > Split and fix TestOzoneShellHA >

[jira] [Resolved] (HDDS-11714) resetDeletedBlockRetryCount with --all may fail and can cause long db lock in large cluster

2025-02-03 Thread Nandakumar (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandakumar resolved HDDS-11714. --- Fix Version/s: 2.0.0 Resolution: Fixed The PR is merged! Thanks [~aryangupta1998] for the con

Re: [PR] HDDS-11714. resetDeletedBlockRetryCount with --all may fail and can cause long db lock in large cluster [ozone]

2025-02-03 Thread via GitHub
nandakumar131 commented on PR #7665: URL: https://github.com/apache/ozone/pull/7665#issuecomment-2633017115 Thanks @aryangupta1998 for the contribution. Thanks @sadanand48 for the review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] HDDS-11714. resetDeletedBlockRetryCount with --all may fail and can cause long db lock in large cluster [ozone]

2025-02-03 Thread via GitHub
nandakumar131 merged PR #7665: URL: https://github.com/apache/ozone/pull/7665 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.a

Re: [PR] HDDS-12186. (addendum) Avoid array allocation for table iterator [ozone]

2025-02-03 Thread via GitHub
adoroszlai commented on PR #7799: URL: https://github.com/apache/ozone/pull/7799#issuecomment-2632994766 Thanks @nandakumar131, @sadanand48 for the review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] HDDS-12186. (addendum) Avoid array allocation for table iterator [ozone]

2025-02-03 Thread via GitHub
adoroszlai merged PR #7799: URL: https://github.com/apache/ozone/pull/7799 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.apac

[jira] [Commented] (HDDS-11839) Mismatch in leadership transfer node ID

2025-02-03 Thread Peter Lee (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923583#comment-17923583 ] Peter Lee commented on HDDS-11839: -- The transfer leadership request with new leader id w

Re: [PR] HDDS-12159. Remove redundant seek for rocksDBs [ozone]

2025-02-03 Thread via GitHub
sadanand48 commented on PR #7794: URL: https://github.com/apache/ozone/pull/7794#issuecomment-2632949583 Also isn't the iterator invalid without calling any of the seek methods, I believe this is the reason we have called seek() api's after initialising iterators . Has Rocksdb behaviour cha

Re: [PR] HDDS-12159. Remove redundant seek for rocksDBs [ozone]

2025-02-03 Thread via GitHub
sadanand48 commented on code in PR #7794: URL: https://github.com/apache/ozone/pull/7794#discussion_r1940547764 ## hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java: ## @@ -2142,9 +2139,7 @@ public DeleteKeysResult getPendingDeletionSubFiles

Re: [PR] HDDS-11714. resetDeletedBlockRetryCount with --all may fail and can cause long db lock in large cluster [ozone]

2025-02-03 Thread via GitHub
aryangupta1998 commented on PR #7665: URL: https://github.com/apache/ozone/pull/7665#issuecomment-2632943904 Thanks @nandakumar131, fixed the test case! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] HDDS-12195. Implement skip() in OzoneFSInputStream [ozone]

2025-02-03 Thread via GitHub
adoroszlai commented on PR #7801: URL: https://github.com/apache/ozone/pull/7801#issuecomment-2632932209 Thanks @oneonestar for the patch. Please enable the `build-branch` [workflow in your fork](https://github.com/oneonestar/ozone/actions). -- This is an automated message from the Apach

[jira] [Updated] (HDDS-12192) Split and fix TestOzoneShellHA

2025-02-03 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12192: -- Labels: pull-request-available (was: ) > Split and fix TestOzoneShellHA >

[jira] [Comment Edited] (HDDS-11643) Investigate time based SCM snapshot/transaction flush not working

2025-02-03 Thread Ivan Zlenko (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923573#comment-17923573 ] Ivan Zlenko edited comment on HDDS-11643 at 2/4/25 5:28 AM: [

[PR] HDDS-12192. Fix TestOzoneShellHA and extract set-bucket-encryption test case [ozone]

2025-02-03 Thread via GitHub
adoroszlai opened a new pull request, #7802: URL: https://github.com/apache/ozone/pull/7802 ## What changes were proposed in this pull request? TestOzoneShellHA and TestOzoneShellHAWithFSO are not run due `ClassNotFoundException` when starting MiniKMS (HDDS-11879). KMS is required f

[jira] [Comment Edited] (HDDS-11643) Investigate time based SCM snapshot/transaction flush not working

2025-02-03 Thread Ivan Zlenko (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923573#comment-17923573 ] Ivan Zlenko edited comment on HDDS-11643 at 2/4/25 5:28 AM: [

[jira] [Commented] (HDDS-11643) Investigate time based SCM snapshot/transaction flush not working

2025-02-03 Thread Ivan Zlenko (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923573#comment-17923573 ] Ivan Zlenko commented on HDDS-11643: [~erose] the old way to delete transaction relie

[jira] [Updated] (HDDS-12195) Missing skip() Method Causes Performance Issues in OzoneFSInputStream

2025-02-03 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12195: -- Labels: pull-request-available (was: ) > Missing skip() Method Causes Performance Issues in Oz

[jira] [Created] (HDDS-12195) Missing skip() Method Causes Performance Issues in OzoneFSInputStream

2025-02-03 Thread Star Poon (Jira)
Star Poon created HDDS-12195: Summary: Missing skip() Method Causes Performance Issues in OzoneFSInputStream Key: HDDS-12195 URL: https://issues.apache.org/jira/browse/HDDS-12195 Project: Apache Ozone

[jira] [Created] (HDDS-12194) Distinguish update/purge snapshot

2025-02-03 Thread Venkat Sambath (Jira)
Venkat Sambath created HDDS-12194: - Summary: Distinguish update/purge snapshot Key: HDDS-12194 URL: https://issues.apache.org/jira/browse/HDDS-12194 Project: Apache Ozone Issue Type: Improvem

[jira] [Commented] (HDDS-12193) Prometheus blocking installSnapshotFromLeader on follower node

2025-02-03 Thread Hemant Kumar (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923524#comment-17923524 ] Hemant Kumar commented on HDDS-12193: - Another similar instance where OMDoubleBufferF

Re: [PR] HDDS-11898. design doc leader side execution [ozone]

2025-02-03 Thread via GitHub
errose28 commented on code in PR #7583: URL: https://github.com/apache/ozone/pull/7583#discussion_r1940247445 ## hadoop-hdds/docs/content/design/leader-execution/leader-execution.md: ## @@ -0,0 +1,477 @@ +--- +title: Ozone Leader Side Execution +summary: Ozone request execution

[jira] [Comment Edited] (HDDS-12182) Error for building project on Windows Subsystem for Linux(WSL)

2025-02-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923505#comment-17923505 ] Wei-Chiu Chuang edited comment on HDDS-12182 at 2/3/25 10:55 PM: --

[jira] [Commented] (HDDS-12182) Error for building project on Windows Subsystem for Linux(WSL)

2025-02-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923505#comment-17923505 ] Wei-Chiu Chuang commented on HDDS-12182: Thanks! Looks like RocksDB is not able

[jira] [Commented] (HDDS-12182) Error for building project on Windows Subsystem for Linux(WSL)

2025-02-03 Thread Sylvia Chao (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923504#comment-17923504 ] Sylvia Chao commented on HDDS-12182: Hello [~weichiu], Apologies for the previous mi

Re: [PR] HDDS-11898. design doc leader side execution [ozone]

2025-02-03 Thread via GitHub
errose28 commented on code in PR #7583: URL: https://github.com/apache/ozone/pull/7583#discussion_r1940209737 ## hadoop-hdds/docs/content/design/leader-execution/leader-execution.md: ## @@ -0,0 +1,477 @@ +--- +title: Ozone Leader Side Execution +summary: Ozone request execution

[jira] [Updated] (HDDS-12182) Error for building project on Windows Subsystem for Linux(WSL)

2025-02-03 Thread Sylvia Chao (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvia Chao updated HDDS-12182: --- Attachment: TEST-org.apache.hadoop.ozone.container.common.TestSchemaOneBackwardsCompatibility.zip >

[jira] [Updated] (HDDS-12193) Prometheus blocking installSnapshotFromLeader on follower node

2025-02-03 Thread Hemant Kumar (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hemant Kumar updated HDDS-12193: Description: Prometheus blocking installSnapshotFromLeader thread can causing delay in bootstrappi

[jira] [Updated] (HDDS-12193) Prometheus blocking installSnapshotFromLeader on follower node

2025-02-03 Thread Hemant Kumar (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hemant Kumar updated HDDS-12193: Attachment: ombootstrapjstack5 > Prometheus blocking installSnapshotFromLeader on follower node > -

[jira] [Updated] (HDDS-12193) Prometheus blocking installSnapshotFromLeader on follower node

2025-02-03 Thread Hemant Kumar (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hemant Kumar updated HDDS-12193: Attachment: ombootstrapjstack2 ombootstrapjstack3 ombootstrapjstack4

[jira] [Created] (HDDS-12193) Prometheus blocking installSnapshotFromLeader on follower node

2025-02-03 Thread Hemant Kumar (Jira)
Hemant Kumar created HDDS-12193: --- Summary: Prometheus blocking installSnapshotFromLeader on follower node Key: HDDS-12193 URL: https://issues.apache.org/jira/browse/HDDS-12193 Project: Apache Ozone

[PR] HDDS-12178. Expand transitive test dependencies [ozone]

2025-02-03 Thread via GitHub
adoroszlai opened a new pull request, #7800: URL: https://github.com/apache/ozone/pull/7800 ## What changes were proposed in this pull request? Some dependencies are added transitively via `hdds-hadoop-dependency-test` and `hdds-test-utils`. This is convenient, but `dependency:analyz

[jira] [Updated] (HDDS-12178) Expand transitive test dependencies

2025-02-03 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12178: -- Labels: pull-request-available (was: ) > Expand transitive test dependencies > ---

Re: [PR] HDDS-12186. (addendum) Avoid array allocation for table iterator [ozone]

2025-02-03 Thread via GitHub
adoroszlai commented on PR #7799: URL: https://github.com/apache/ozone/pull/7799#issuecomment-2632088404 > don't we have any unit test for this? Added, thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Commented] (HDDS-11643) Investigate time based SCM snapshot/transaction flush not working

2025-02-03 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923476#comment-17923476 ] Ethan Rose commented on HDDS-11643: --- Hi [~izlenko] I think the repro would be different

[jira] [Commented] (HDDS-12182) Error for building project on Windows Subsystem for Linux(WSL)

2025-02-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923468#comment-17923468 ] Wei-Chiu Chuang commented on HDDS-12182: Hi [~sylviac7] could you share the test

[jira] [Created] (HDDS-12192) Split and fix TestOzoneShellHA

2025-02-03 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-12192: --- Summary: Split and fix TestOzoneShellHA Key: HDDS-12192 URL: https://issues.apache.org/jira/browse/HDDS-12192 Project: Apache Ozone Issue Type: Sub-tas

[jira] [Created] (HDDS-12191) Create Grafana dashboards for the block preallocation and cache flow in OM

2025-02-03 Thread Tanvi Penumudy (Jira)
Tanvi Penumudy created HDDS-12191: - Summary: Create Grafana dashboards for the block preallocation and cache flow in OM Key: HDDS-12191 URL: https://issues.apache.org/jira/browse/HDDS-12191 Project: A

[jira] [Created] (HDDS-12190) Introduce metrics for the preallocated blocks flow in OM

2025-02-03 Thread Tanvi Penumudy (Jira)
Tanvi Penumudy created HDDS-12190: - Summary: Introduce metrics for the preallocated blocks flow in OM Key: HDDS-12190 URL: https://issues.apache.org/jira/browse/HDDS-12190 Project: Apache Ozone

[jira] [Created] (HDDS-12189) Improve RocksDB compaction in Ozone

2025-02-03 Thread Wei-Chiu Chuang (Jira)
Wei-Chiu Chuang created HDDS-12189: -- Summary: Improve RocksDB compaction in Ozone Key: HDDS-12189 URL: https://issues.apache.org/jira/browse/HDDS-12189 Project: Apache Ozone Issue Type: Epic

Re: [PR] HDDS-12180. Store snapshot in CachingSpaceUsageSource [ozone]

2025-02-03 Thread via GitHub
adoroszlai commented on code in PR #7798: URL: https://github.com/apache/ozone/pull/7798#discussion_r1939858831 ## hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/fs/CachingSpaceUsageSource.java: ## @@ -104,7 +105,15 @@ public long getUsedSpace() { @Override public

Re: [PR] HDDS-12180. Store snapshot in CachingSpaceUsageSource [ozone]

2025-02-03 Thread via GitHub
kerneltime commented on code in PR #7798: URL: https://github.com/apache/ozone/pull/7798#discussion_r1939846759 ## hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/fs/CachingSpaceUsageSource.java: ## @@ -104,7 +105,15 @@ public long getUsedSpace() { @Override public

[jira] [Commented] (HDDS-12047) Merge TestOzoneClientMultipartUploadWithFSO to OzoneRpcClientTests

2025-02-03 Thread Ivan Andika (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923387#comment-17923387 ] Ivan Andika commented on HDDS-12047: > I’m planning to merge the tests into TestOzone

[jira] [Comment Edited] (HDDS-12047) Merge TestOzoneClientMultipartUploadWithFSO to OzoneRpcClientTests

2025-02-03 Thread Peter Lee (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923388#comment-17923388 ] Peter Lee edited comment on HDDS-12047 at 2/3/25 5:19 PM: -- > Do

[jira] [Commented] (HDDS-12047) Merge TestOzoneClientMultipartUploadWithFSO to OzoneRpcClientTests

2025-02-03 Thread Peter Lee (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923388#comment-17923388 ] Peter Lee commented on HDDS-12047: -- > Do you mean to just rename the TestOzoneClientMult

[jira] [Comment Edited] (HDDS-12047) Merge TestOzoneClientMultipartUploadWithFSO to OzoneRpcClientTests

2025-02-03 Thread Peter Lee (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923383#comment-17923383 ] Peter Lee edited comment on HDDS-12047 at 2/3/25 5:16 PM: -- I’m p

[jira] [Comment Edited] (HDDS-12047) Merge TestOzoneClientMultipartUploadWithFSO to OzoneRpcClientTests

2025-02-03 Thread Ivan Andika (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923387#comment-17923387 ] Ivan Andika edited comment on HDDS-12047 at 2/3/25 5:14 PM: >

[jira] [Commented] (HDDS-12047) Merge TestOzoneClientMultipartUploadWithFSO to OzoneRpcClientTests

2025-02-03 Thread Peter Lee (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923383#comment-17923383 ] Peter Lee commented on HDDS-12047: -- I’m planning to merge the tests into TestOzoneClient

[PR] HDDS-12186. (addendum) Avoid array allocation for table iterator [ozone]

2025-02-03 Thread via GitHub
adoroszlai opened a new pull request, #7799: URL: https://github.com/apache/ozone/pull/7799 ## What changes were proposed in this pull request? Addendum for #7797. Get raw size from value, not iterator, which always returns 0. https://issues.apache.org/jira/browse/HDDS-12186

[jira] [Updated] (HDDS-12180) Store snapshot in CachingSpaceUsageSource

2025-02-03 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-12180: Status: Patch Available (was: In Progress) > Store snapshot in CachingSpaceUsageSource > -

[jira] [Resolved] (HDDS-12186) Avoid array allocation for table iterator

2025-02-03 Thread Nandakumar (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandakumar resolved HDDS-12186. --- Fix Version/s: 2.0.0 Resolution: Fixed The PR is merged! Thanks [~sshenoy] for the contributi

Re: [PR] HDDS-12186. Avoid array allocation for table iterator. [ozone]

2025-02-03 Thread via GitHub
nandakumar131 commented on PR #7797: URL: https://github.com/apache/ozone/pull/7797#issuecomment-2631417643 Thanks @sadanand48 for the contribution & thanks @adoroszlai for the review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] HDDS-12186. Avoid array allocation for table iterator. [ozone]

2025-02-03 Thread via GitHub
nandakumar131 merged PR #7797: URL: https://github.com/apache/ozone/pull/7797 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.a

[jira] [Created] (HDDS-12188) Move server-only classes from hdds-common to hdds-server-framework

2025-02-03 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-12188: --- Summary: Move server-only classes from hdds-common to hdds-server-framework Key: HDDS-12188 URL: https://issues.apache.org/jira/browse/HDDS-12188 Project: Apach

[jira] [Created] (HDDS-12187) Use hadoop-dependency modules as parent, not dependency

2025-02-03 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-12187: --- Summary: Use hadoop-dependency modules as parent, not dependency Key: HDDS-12187 URL: https://issues.apache.org/jira/browse/HDDS-12187 Project: Apache Ozone

Re: [PR] HDDS-11442. Add dashboard for memory consumption metrics [ozone]

2025-02-03 Thread via GitHub
dombizita commented on PR #7198: URL: https://github.com/apache/ozone/pull/7198#issuecomment-2630811581 /ready -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

[jira] [Assigned] (HDDS-12185) Enhance FileSizeCountTask for Faster Processing

2025-02-03 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai reassigned HDDS-12185: --- Assignee: Arafat Khan > Enhance FileSizeCountTask for Faster Processing > --

[jira] [Commented] (HDDS-10823) Ozone Ranger plugin does recursive listing for ACL checks

2025-02-03 Thread Zita Dombi (Jira)
[ https://issues.apache.org/jira/browse/HDDS-10823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923243#comment-17923243 ] Zita Dombi commented on HDDS-10823: --- [~swamirishi] The Ranger side support for this was

[jira] [Updated] (HDDS-12180) Store snapshot in CachingSpaceUsageSource

2025-02-03 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12180: -- Labels: pull-request-available (was: ) > Store snapshot in CachingSpaceUsageSource > -

[PR] HDDS-12180. Store snapshot in CachingSpaceUsageSource [ozone]

2025-02-03 Thread via GitHub
adoroszlai opened a new pull request, #7798: URL: https://github.com/apache/ozone/pull/7798 ## What changes were proposed in this pull request? `CachingSpaceUsageSource` may benefit from storing `SpaceUsageSource#snapshot` until individual values change, to reduce object creation.

Re: [PR] HDDS-12186. Avoid array allocation for table iterator. [ozone]

2025-02-03 Thread via GitHub
adoroszlai commented on code in PR #7797: URL: https://github.com/apache/ozone/pull/7797#discussion_r1939026921 ## hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/Table.java: ## @@ -337,12 +337,8 @@ interface KeyValue { VALUE getValue() throws IOExcept

[PR] HDDS-12186. Avoid array allocation for table iterator. [ozone]

2025-02-03 Thread via GitHub
sadanand48 opened a new pull request, #7797: URL: https://github.com/apache/ozone/pull/7797 ## What changes were proposed in this pull request? Changes in [HDDS-11508](https://issues.apache.org/jira/browse/HDDS-11508) [cause](https://github.com/apache/ozone/pull/7365#pullrequestreview-258

Re: [PR] HDDS-12184. Enhance NSSummaryTask process() for Faster Processing. [ozone]

2025-02-03 Thread via GitHub
adoroszlai commented on code in PR #7795: URL: https://github.com/apache/ozone/pull/7795#discussion_r1938998104 ## hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/NSSummaryTask.java: ## @@ -101,21 +101,37 @@ public String getTaskName() { @Override publ

[jira] [Updated] (HDDS-12186) Avoid array allocation for table iterator

2025-02-03 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12186: -- Labels: pull-request-available (was: ) > Avoid array allocation for table iterator > -

[jira] [Updated] (HDDS-12185) Enhance FileSizeCountTask for Faster Processing

2025-02-03 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12185: -- Labels: pull-request-available (was: ) > Enhance FileSizeCountTask for Faster Processing > ---

[PR] HDDS-12185. Enhance FileSizeCountTask for Faster Processing. [ozone]

2025-02-03 Thread via GitHub
ArafatKhan2198 opened a new pull request, #7796: URL: https://github.com/apache/ozone/pull/7796 ## What changes were proposed in this pull request? Optimize reprocess() by: Running parallel threads for each BucketLayout. Using thread-local maps to accumulate (container → key)

[jira] [Comment Edited] (HDDS-11585) Add DN Ratis log purge parameters to close pipeline with slow follower

2025-02-03 Thread Ivan Andika (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923209#comment-17923209 ] Ivan Andika edited comment on HDDS-11585 at 2/3/25 8:22 AM: C

[jira] [Resolved] (HDDS-11585) Add DN Ratis log purge parameters to close pipeline with slow follower

2025-02-03 Thread Ivan Andika (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Andika resolved HDDS-11585. Resolution: Won't Do > Add DN Ratis log purge parameters to close pipeline with slow follower > ---

[jira] [Assigned] (HDDS-12186) Avoid array allocation for table iterator

2025-02-03 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai reassigned HDDS-12186: --- Assignee: Sadanand Shenoy > Avoid array allocation for table iterator >

[jira] [Created] (HDDS-12186) Avoid array allocation for table iterator

2025-02-03 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-12186: --- Summary: Avoid array allocation for table iterator Key: HDDS-12186 URL: https://issues.apache.org/jira/browse/HDDS-12186 Project: Apache Ozone Issue Ty

[jira] [Commented] (HDDS-11585) Add DN Ratis log purge parameters to close pipeline with slow follower

2025-02-03 Thread Ivan Andika (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923209#comment-17923209 ] Ivan Andika commented on HDDS-11585: Closing this for now. Automatically closing the

Re: [PR] HDDS-11508. Decouple delete batch limits from Ratis request size for DirectoryDeletingService. [ozone]

2025-02-03 Thread via GitHub
adoroszlai commented on PR #7365: URL: https://github.com/apache/ozone/pull/7365#issuecomment-2630258844 Created HDDS-12186 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] HDDS-11585. Add DN Ratis log purge parameters to close pipeline with slow follower [ozone]

2025-02-03 Thread via GitHub
ivandika3 closed pull request #7319: HDDS-11585. Add DN Ratis log purge parameters to close pipeline with slow follower URL: https://github.com/apache/ozone/pull/7319 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] HDDS-11585. Add DN Ratis log purge parameters to close pipeline with slow follower [ozone]

2025-02-03 Thread via GitHub
ivandika3 commented on PR #7319: URL: https://github.com/apache/ozone/pull/7319#issuecomment-2630256849 Closing this for now. Automatically closing the pipeline for slow follower might hide the underlying write degradation issue and might cause pipelines to be created and closed fre

[jira] [Created] (HDDS-12185) Enhance FileSizeCountTask for Faster Processing

2025-02-03 Thread Arafat Khan (Jira)
Arafat Khan created HDDS-12185: -- Summary: Enhance FileSizeCountTask for Faster Processing Key: HDDS-12185 URL: https://issues.apache.org/jira/browse/HDDS-12185 Project: Apache Ozone Issue Type:

Re: [PR] HDDS-11508. Decouple delete batch limits from Ratis request size for DirectoryDeletingService. [ozone]

2025-02-03 Thread via GitHub
sadanand48 commented on PR #7365: URL: https://github.com/apache/ozone/pull/7365#issuecomment-2630253126 Sure thanks @adoroszlai , will open a jira to handle this and raise a PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[jira] [Updated] (HDDS-12159) Remove unnecessary iterator seek in fileTable and DirectoryTable uses

2025-02-03 Thread Tejaskriya Madhan (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tejaskriya Madhan updated HDDS-12159: - Summary: Remove unnecessary iterator seek in fileTable and DirectoryTable uses (was: Rem

[jira] [Resolved] (HDDS-11508) Decouple delete batch limits from Ratis request size

2025-02-03 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-11508. - Fix Version/s: 2.0.0 Resolution: Implemented > Decouple delete batch limits from R

[jira] [Updated] (HDDS-12184) Enhance NSSummaryTask process() for Faster Processing

2025-02-03 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12184: -- Labels: pull-request-available (was: ) > Enhance NSSummaryTask process() for Faster Processing

[jira] [Updated] (HDDS-12159) Remove unnecessary iterator seek in OMFileRequest#checkSubFileExists

2025-02-03 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12159: -- Labels: pull-request-available (was: ) > Remove unnecessary iterator seek in OMFileRequest#che

[PR] HDDS-12159. Remove redundant seek for rocksDBs [ozone]

2025-02-03 Thread via GitHub
Tejaskriya opened a new pull request, #7794: URL: https://github.com/apache/ozone/pull/7794 ## What changes were proposed in this pull request? In many code bits, an iterator is created for a rocksDB table. During this, a seek to the first record takes place internally. Following whic