[jira] [Comment Edited] (HDDS-9377) Call MutableStat.add(long x) method as a runnable task in a separate service executor

2025-02-04 Thread Alex Juncevich (Jira)
[ https://issues.apache.org/jira/browse/HDDS-9377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923947#comment-17923947 ] Alex Juncevich edited comment on HDDS-9377 at 2/5/25 7:51 AM: --

[jira] [Commented] (HDDS-9377) Call MutableStat.add(long x) method as a runnable task in a separate service executor

2025-02-04 Thread Alex Juncevich (Jira)
[ https://issues.apache.org/jira/browse/HDDS-9377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923947#comment-17923947 ] Alex Juncevich commented on HDDS-9377: -- Without a fix, it was a 3000-3200 writing ope

Re: [PR] HDDS-12159. Remove redundant seek for rocksDBs [ozone]

2025-02-04 Thread via GitHub
nandakumar131 commented on code in PR #7794: URL: https://github.com/apache/ozone/pull/7794#discussion_r1942370337 ## hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmMetadataManagerImpl.java: ## @@ -1955,19 +1955,15 @@ public Set getMultipartUploadKeys(

[jira] [Updated] (HDDS-12202) OpsCreate and OpsAppend metrics not incremented

2025-02-04 Thread Peter Lee (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Lee updated HDDS-12202: - Status: Patch Available (was: Open) > OpsCreate and OpsAppend metrics not incremented >

[jira] (HDDS-4895) Add integration test for SequenceIdGen

2025-02-04 Thread Peter Lee (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4895 ] Peter Lee deleted comment on HDDS-4895: - was (Author: JIRAUSER306303): [~glengeng] Is this covered by TestSequenceIDGenerator? > Add integration test for SequenceIdGen > --

Re: [PR] HDDS-12200.fixed grammatical changes in the following documents - OM High Availability ,Ozone Erasure Coding ,Ozone Snapshot [ozone]

2025-02-04 Thread via GitHub
Tejaskriya commented on PR #7806: URL: https://github.com/apache/ozone/pull/7806#issuecomment-2635714805 @sreejasahithi could you change the title of the PR to have a space after the jira id, like below: `HDDS-12200. fixed grammatical changes in the following documents - OM High Availabi

[jira] [Assigned] (HDDS-10607) Remove ozone.block.deleting.container.limit.per.interval property as it's not used anymore

2025-02-04 Thread Sreeja (Jira)
[ https://issues.apache.org/jira/browse/HDDS-10607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreeja reassigned HDDS-10607: - Assignee: Sreeja (was: David) > Remove ozone.block.deleting.container.limit.per.interval property as it

[jira] [Commented] (HDDS-12194) Distinguish update/purge snapshot

2025-02-04 Thread Venkat Sambath (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923910#comment-17923910 ] Venkat Sambath commented on HDDS-12194: --- I tested in 1.4.1 and I no longer see the

[jira] [Resolved] (HDDS-12194) Distinguish update/purge snapshot

2025-02-04 Thread Venkat Sambath (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkat Sambath resolved HDDS-12194. --- Resolution: Cannot Reproduce > Distinguish update/purge snapshot > --

[jira] [Updated] (HDDS-12194) Distinguish update/purge snapshot

2025-02-04 Thread Venkat Sambath (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkat Sambath updated HDDS-12194: -- Description: Currently when snapshot is created/deleted in ozone-ha env, ozone-om logs indicat

[jira] [Updated] (HDDS-12211) Add TestHttpFSMetrics

2025-02-04 Thread Peter Lee (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Lee updated HDDS-12211: - Description: Add TestHttpFSMetrics https://github.com/apache/ozone/pull/7811#issuecomment-2635150430

Re: [PR] HDDS-12202. OpsCreate and OpsAppend metrics not incremented [ozone]

2025-02-04 Thread via GitHub
peterxcli commented on PR #7811: URL: https://github.com/apache/ozone/pull/7811#issuecomment-2635582291 Thanks @jojochuang for reviewing, created https://issues.apache.org/jira/browse/HDDS-12211 to address the test. -- This is an automated message from the Apache Git Service. To respond t

[jira] [Assigned] (HDDS-12211) Add TestHttpFSMetrics

2025-02-04 Thread Peter Lee (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Lee reassigned HDDS-12211: Assignee: Peter Lee > Add TestHttpFSMetrics > - > > Key: HDDS-

[jira] [Created] (HDDS-12211) Add TestHttpFSMetrics

2025-02-04 Thread Peter Lee (Jira)
Peter Lee created HDDS-12211: Summary: Add TestHttpFSMetrics Key: HDDS-12211 URL: https://issues.apache.org/jira/browse/HDDS-12211 Project: Apache Ozone Issue Type: Test Reporter: Pet

Re: [PR] HDDS-10220. Add test case to cover HDDS-9930 batch key deletion [ozone]

2025-02-04 Thread via GitHub
jojochuang commented on code in PR #7792: URL: https://github.com/apache/ozone/pull/7792#discussion_r1942167819 ## hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestHSync.java: ## @@ -583,6 +583,55 @@ public void testHSyncDeletedKey() throws Exception {

[jira] [Updated] (HDDS-12151) Check every single write operation on an Ozone datanode is handled correctly in the case of full volumes

2025-02-04 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDDS-12151: -- Description: We observed cases where full Ozone volumes (reserved space also gone. StorageVolumeChecke

Re: [PR] HDDS-12203. Initialize block length before skip [ozone]

2025-02-04 Thread via GitHub
oneonestar commented on PR #7809: URL: https://github.com/apache/ozone/pull/7809#issuecomment-2635514930 @jojochuang @adoroszlai Thanks for the quick review =) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[jira] [Commented] (HDDS-10509) Allow running ratis shell commands in secure Ozone cluster.

2025-02-04 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-10509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923879#comment-17923879 ] Tsz-wo Sze commented on HDDS-10509: --- Could we do Kerberos over TLS? - See https://data

[jira] [Created] (HDDS-12210) Tarball creation interfering with snapshot purge

2025-02-04 Thread Hemant Kumar (Jira)
Hemant Kumar created HDDS-12210: --- Summary: Tarball creation interfering with snapshot purge Key: HDDS-12210 URL: https://issues.apache.org/jira/browse/HDDS-12210 Project: Apache Ozone Issue Ty

[jira] [Created] (HDDS-12209) Make replica verification multi-threaded

2025-02-04 Thread Ethan Rose (Jira)
Ethan Rose created HDDS-12209: - Summary: Make replica verification multi-threaded Key: HDDS-12209 URL: https://issues.apache.org/jira/browse/HDDS-12209 Project: Apache Ozone Issue Type: Sub-task

[jira] [Created] (HDDS-12208) Add option for `ozone debug replicas verify` to force OM container cache refresh

2025-02-04 Thread Ethan Rose (Jira)
Ethan Rose created HDDS-12208: - Summary: Add option for `ozone debug replicas verify` to force OM container cache refresh Key: HDDS-12208 URL: https://issues.apache.org/jira/browse/HDDS-12208 Project: Apa

Re: [PR] HDDS-11891. Design doc for find Block missing Key tool [ozone]

2025-02-04 Thread via GitHub
errose28 commented on PR #7548: URL: https://github.com/apache/ozone/pull/7548#issuecomment-2635412705 I like the idea of being able to quickly check block existence without downloading all the data. We are working on a similar set of tools that might be useful in HDDS-12206. The final impl

[jira] [Assigned] (HDDS-12206) Create an `ozone debug` command to run checks on all replicas

2025-02-04 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Rose reassigned HDDS-12206: - Assignee: Rishabh Patel > Create an `ozone debug` command to run checks on all replicas > --

[jira] [Created] (HDDS-12207) Unify output of `ozone debug replicas verify` checks

2025-02-04 Thread Ethan Rose (Jira)
Ethan Rose created HDDS-12207: - Summary: Unify output of `ozone debug replicas verify` checks Key: HDDS-12207 URL: https://issues.apache.org/jira/browse/HDDS-12207 Project: Apache Ozone Issue Typ

[jira] [Updated] (HDDS-12057) Create initial `ozone debug replicas verify` command with the ability to verify checksums

2025-02-04 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Rose updated HDDS-12057: -- Status: Patch Available (was: Open) > Create initial `ozone debug replicas verify` command with the ab

[jira] [Updated] (HDDS-12057) Create initial `ozone debug replicas verify` command with the ability to verify checksums

2025-02-04 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Rose updated HDDS-12057: -- Parent: HDDS-12206 Issue Type: Sub-task (was: Improvement) > Create initial `ozone debug repli

[jira] [Updated] (HDDS-12119) Move find-missing-padding under ozone debug replicas verify command

2025-02-04 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Rose updated HDDS-12119: -- Parent Issue: HDDS-12206 (was: HDDS-12057) > Move find-missing-padding under ozone debug replicas veri

[jira] [Updated] (HDDS-12118) Remove SCM dependency from find-missing-padding debug command

2025-02-04 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Rose updated HDDS-12118: -- Parent Issue: HDDS-12206 (was: HDDS-12057) > Remove SCM dependency from find-missing-padding debug com

[jira] [Updated] (HDDS-12057) Create initial `ozone debug replicas verify` command with the ability to verify checksums

2025-02-04 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Rose updated HDDS-12057: -- Summary: Create initial `ozone debug replicas verify` command with the ability to verify checksums (wa

[jira] [Created] (HDDS-12206) Create an `ozone debug` command to run checks on all replicas

2025-02-04 Thread Ethan Rose (Jira)
Ethan Rose created HDDS-12206: - Summary: Create an `ozone debug` command to run checks on all replicas Key: HDDS-12206 URL: https://issues.apache.org/jira/browse/HDDS-12206 Project: Apache Ozone

Re: [PR] HDDS-11866. Remove code paths for non-Ratis OM [ozone]

2025-02-04 Thread via GitHub
jojochuang commented on PR #7778: URL: https://github.com/apache/ozone/pull/7778#issuecomment-2635349114 Would an upgrade from non-HA OM to HA OM? Not sure if the upgrade path is tested properly -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] HDDS-11866. Remove code paths for non-Ratis OM [ozone]

2025-02-04 Thread via GitHub
jojochuang commented on code in PR #7778: URL: https://github.com/apache/ozone/pull/7778#discussion_r1942052873 ## hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java: ## @@ -4289,8 +4249,9 @@ public void checkLeaderStatus() throws OMNotLeaderE

[jira] [Updated] (HDDS-12118) Remove SCM dependency from find-missing-padding debug command

2025-02-04 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Rose updated HDDS-12118: -- Parent Issue: HDDS-12057 (was: HDDS-11944) > Remove SCM dependency from find-missing-padding debug com

[jira] [Updated] (HDDS-12119) Move find-missing-padding under ozone debug replicas verify command

2025-02-04 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Rose updated HDDS-12119: -- Parent Issue: HDDS-12057 (was: HDDS-11944) > Move find-missing-padding under ozone debug replicas veri

[jira] [Updated] (HDDS-12057) Create `ozone debug replicas verify` command to verify checksums of all block data

2025-02-04 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Rose updated HDDS-12057: -- Parent: (was: HDDS-11944) Issue Type: Improvement (was: Sub-task) > Create `ozone debug re

Re: [PR] HDDS-12159. Remove redundant seek for rocksDBs [ozone]

2025-02-04 Thread via GitHub
jojochuang commented on code in PR #7794: URL: https://github.com/apache/ozone/pull/7794#discussion_r1942028422 ## hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmMetadataManagerImpl.java: ## @@ -1955,19 +1955,15 @@ public Set getMultipartUploadKeys(

[jira] [Commented] (HDDS-12159) Remove unnecessary iterator seek in fileTable and DirectoryTable uses

2025-02-04 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923859#comment-17923859 ] Wei-Chiu Chuang commented on HDDS-12159: Stacetrace for posterity:   {noformat}

[jira] [Created] (HDDS-12205) Reduce log level in TestHSync

2025-02-04 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-12205: --- Summary: Reduce log level in TestHSync Key: HDDS-12205 URL: https://issues.apache.org/jira/browse/HDDS-12205 Project: Apache Ozone Issue Type: Task

Re: [PR] [DRAFT] Raft group for each bucket [ozone]

2025-02-04 Thread via GitHub
kerneltime commented on PR #7808: URL: https://github.com/apache/ozone/pull/7808#issuecomment-2635218964 This is an idea discussed multiple times to increase the OM bandwidth for update APIs by leveraging multi Ratis. Thank you for initiating this. Did you get any performance numbers? If yo

Re: [PR] HDDS-12097. Enhance Container Key Mapper for Faster Processing. [ozone]

2025-02-04 Thread via GitHub
kerneltime commented on code in PR #7770: URL: https://github.com/apache/ozone/pull/7770#discussion_r1941979706 ## hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/ReconContainerMetadataManagerImpl.java: ## @@ -108,16 +108,23 @@ public ReconContainerMetada

Re: [PR] HDDS-12203. Initialize block length before skip [ozone]

2025-02-04 Thread via GitHub
adoroszlai commented on PR #7809: URL: https://github.com/apache/ozone/pull/7809#issuecomment-2635175315 Thanks @oneonestar for the patch, @jojochuang for the review. Please trigger CI by "Approve and run" before merge. (Unfortunately the `labeler` workflow runs without approval, adding a

Re: [PR] HDDS-12183. Reuse cluster across safe test classes [ozone]

2025-02-04 Thread via GitHub
adoroszlai commented on PR #7793: URL: https://github.com/apache/ozone/pull/7793#issuecomment-2635150554 Thanks @sodonnel for the review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] HDDS-12202. OpsCreate and OpsAppend metrics not incremented [ozone]

2025-02-04 Thread via GitHub
jojochuang commented on PR #7811: URL: https://github.com/apache/ozone/pull/7811#issuecomment-2635150430 Please consider creating a test class to check httpfs server metrics, similar to how TestS3GatewayMetrics does the same for s3 gateway. Can be a new jira of course. -- This is an auto

[jira] [Resolved] (HDDS-12203) skip() in MultipartInputStream doesn't call initialize()

2025-02-04 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDDS-12203. Fix Version/s: 2.0.0 Resolution: Fixed > skip() in MultipartInputStream doesn't call

Re: [PR] HDDS-12203. Initialize block length before skip [ozone]

2025-02-04 Thread via GitHub
jojochuang commented on PR #7809: URL: https://github.com/apache/ozone/pull/7809#issuecomment-2635156980 Merged. Thanks @oneonestar welcome to the Ozone community! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] HDDS-12203. Initialize block length before skip [ozone]

2025-02-04 Thread via GitHub
jojochuang merged PR #7809: URL: https://github.com/apache/ozone/pull/7809 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.apac

[jira] [Updated] (HDDS-12183) Reuse cluster across safe test classes

2025-02-04 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-12183: Fix Version/s: 2.0.0 Resolution: Implemented Status: Resolved (was: Patch

Re: [PR] HDDS-12183. Reuse cluster across safe test classes [ozone]

2025-02-04 Thread via GitHub
adoroszlai merged PR #7793: URL: https://github.com/apache/ozone/pull/7793 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.apac

[jira] [Updated] (HDDS-12097) Enhance Container Key Mapper for Faster Processing

2025-02-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12097: -- Labels: pull-request-available (was: ) > Enhance Container Key Mapper for Faster Processing >

[jira] [Comment Edited] (HDDS-11777) Create common method for tasks that need to iterate OM namespace in parallel

2025-02-04 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17902741#comment-17902741 ] Ethan Rose edited comment on HDDS-11777 at 2/4/25 9:33 PM: --- In

Re: [PR] HDDS-12097. Enhance Container Key Mapper for Faster Processing. [ozone]

2025-02-04 Thread via GitHub
errose28 commented on PR #7770: URL: https://github.com/apache/ozone/pull/7770#issuecomment-2635139762 How does Recon get the full path to the key for FSO? Currently it looks like it will only work on the key with parent IDs. I think we need something like HDDS-11777 for this. -- This is

[jira] [Updated] (HDDS-11777) Create common method for tasks that need to iterate OM namespace in parallel

2025-02-04 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Rose updated HDDS-11777: -- Summary: Create common method for tasks that need to iterate OM namespace in parallel (was: Common cod

[jira] [Updated] (HDDS-11777) Common code for reverse mapping db

2025-02-04 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Rose updated HDDS-11777: -- Description: Add a utility method that supports iterating the filesystem tree from RocksDB in a parall

Re: [PR] HDDS-9377. Implement some metrics with RW-lock instead of synchronization [ozone]

2025-02-04 Thread via GitHub
juncevich commented on PR #7708: URL: https://github.com/apache/ozone/pull/7708#issuecomment-2635034797 I looked at communication in Hadoop(mailing lists) and discussion and realization can take a long time ) I think the best way is not to depend on Hadoop libraries and change the metric

[jira] [Created] (HDDS-12204) Improve failover logging

2025-02-04 Thread Wei-Chiu Chuang (Jira)
Wei-Chiu Chuang created HDDS-12204: -- Summary: Improve failover logging Key: HDDS-12204 URL: https://issues.apache.org/jira/browse/HDDS-12204 Project: Apache Ozone Issue Type: Improvement

[jira] [Resolved] (HDDS-10734) [Hbase Ozone] ImportTSV fails during OM Rolling Restart with "SecretManager$InvalidToken: Tampered/Invalid token."

2025-02-04 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDDS-10734. Resolution: Duplicate > [Hbase Ozone] ImportTSV fails during OM Rolling Restart with > "Se

[jira] [Updated] (HDDS-12202) OpsCreate and OpsAppend metrics not incremented

2025-02-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12202: -- Labels: pull-request-available (was: ) > OpsCreate and OpsAppend metrics not incremented > ---

[PR] HDDS-12202. OpsCreate and OpsAppend metrics not incremented [ozone]

2025-02-04 Thread via GitHub
peterxcli opened a new pull request, #7811: URL: https://github.com/apache/ozone/pull/7811 ## What changes were proposed in this pull request? FSCreate and FSAppend only update bytesWritten, but not opsCreate and opsAppend counters. ## What is the link to the Apache JIRA

[jira] [Commented] (HDDS-11839) Mismatch in leadership transfer node ID

2025-02-04 Thread Peter Lee (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923776#comment-17923776 ] Peter Lee commented on HDDS-11839: -- I'm still investigating another way to get the mappi

Re: [PR] HDDS-11149. Have a generic version Validator for validating Requests [ozone]

2025-02-04 Thread via GitHub
fapifta commented on PR #6932: URL: https://github.com/apache/ozone/pull/6932#issuecomment-2634517141 @swamirishi I think we should move this back to open, with the last changes it seems to be ok, but we need to solve the conflicts, please let me know if there are any conflicts that are not

Re: [PR] HDDS-11149. Have a generic version Validator for validating Requests [ozone]

2025-02-04 Thread via GitHub
fapifta commented on code in PR #6932: URL: https://github.com/apache/ozone/pull/6932#discussion_r1933354617 ## hadoop-hdds/annotations/src/main/java/org/apache/ozone/annotations/OmRequestFeatureValidatorProcessor.java: ## Review Comment: I kind of disagree here still...

[jira] [Commented] (HDDS-10509) Allow running ratis shell commands in secure Ozone cluster.

2025-02-04 Thread Jira
[ https://issues.apache.org/jira/browse/HDDS-10509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923775#comment-17923775 ] István Fajth commented on HDDS-10509: - The Client to DataNode connection is the only

[jira] [Commented] (HDDS-11839) Mismatch in leadership transfer node ID

2025-02-04 Thread Peter Lee (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923767#comment-17923767 ] Peter Lee commented on HDDS-11839: -- I tried this: {code:java} private String getRaftPeer

[jira] [Assigned] (HDDS-12202) OpsCreate and OpsAppend metrics not incremented

2025-02-04 Thread Peter Lee (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Lee reassigned HDDS-12202: Assignee: Peter Lee > OpsCreate and OpsAppend metrics not incremented > --

Re: [PR] HDDS-12110. Optimize memory overhead for OM background tasks. [ozone]

2025-02-04 Thread via GitHub
devmadhuu commented on code in PR #7743: URL: https://github.com/apache/ozone/pull/7743#discussion_r1941381736 ## hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/tasks/TestOmTableInsightTask.java: ## @@ -154,6 +160,27 @@ private void initializeInjector() throws IO

[jira] [Updated] (HDDS-11953) Ozone Recon - Improve Recon OM sync process based on continuous pull of OM data

2025-02-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-11953: -- Labels: pull-request-available (was: ) > Ozone Recon - Improve Recon OM sync process based on

[PR] HDDS-11953. Ozone Recon - Improve Recon OM sync process based on continuous pull of OM data. [ozone]

2025-02-04 Thread via GitHub
devmadhuu opened a new pull request, #7810: URL: https://github.com/apache/ozone/pull/7810 ## What changes were proposed in this pull request? This PR change is to get rid of interval delay configs for Recon OM DB snapshot sync with active OM DB because these configs may not make much sen

Re: [PR] HDDS-12195. Implement skip() in OzoneFSInputStream [ozone]

2025-02-04 Thread via GitHub
oneonestar commented on PR #7801: URL: https://github.com/apache/ozone/pull/7801#issuecomment-2634019011 I created another jira and PR to fix the underlying cause: * https://issues.apache.org/jira/browse/HDDS-12203 * https://github.com/apache/ozone/pull/7809 -- This is an automated m

[jira] [Updated] (HDDS-12203) skip() in MultipartInputStream doesn't call initialize()

2025-02-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12203: -- Labels: pull-request-available (was: ) > skip() in MultipartInputStream doesn't call initializ

[jira] [Created] (HDDS-12203) skip() in MultipartInputStream doesn't call initialize()

2025-02-04 Thread Star Poon (Jira)
Star Poon created HDDS-12203: Summary: skip() in MultipartInputStream doesn't call initialize() Key: HDDS-12203 URL: https://issues.apache.org/jira/browse/HDDS-12203 Project: Apache Ozone Issue T

[PR] [DRAFT] Raft group for each bucket [ozone]

2025-02-04 Thread via GitHub
juncevich opened a new pull request, #7808: URL: https://github.com/apache/ozone/pull/7808 ## What changes were proposed in this pull request? Test performance improvements when each bucket will have own raft group -- This is an automated message from the Apache Git Service. To

[jira] [Commented] (HDDS-11839) Mismatch in leadership transfer node ID

2025-02-04 Thread Sadanand Shenoy (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923658#comment-17923658 ] Sadanand Shenoy commented on HDDS-11839: {code:java} private void getRaftPeerId(S

[jira] [Created] (HDDS-12202) OpsCreate and OpsAppend metrics not incremented

2025-02-04 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-12202: --- Summary: OpsCreate and OpsAppend metrics not incremented Key: HDDS-12202 URL: https://issues.apache.org/jira/browse/HDDS-12202 Project: Apache Ozone Is

[jira] [Updated] (HDDS-12199) Explanation in Merge Container RocksDB in DN Documentation is not clear

2025-02-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12199: -- Labels: pull-request-available (was: ) > Explanation in Merge Container RocksDB in DN Document

[PR] HDDS-12199. Update in explanation in Merge Container RocksDB in DN Documentation [ozone]

2025-02-04 Thread via GitHub
Gargi-jais11 opened a new pull request, #7807: URL: https://github.com/apache/ozone/pull/7807 ## What changes were proposed in this pull request? In the Merge Container RocksDB in DN Documentation in The third paragraph "Unlike the previous approach, this “Merge Container RocksDB

[jira] [Updated] (HDDS-12200) grammatical changes in the following documents - OM High Availability ,Ozone Erasure Coding ,Ozone Snapshot

2025-02-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12200: -- Labels: pull-request-available (was: ) > grammatical changes in the following documents - OM H

[PR] HDDS-12200.fixed grammatical changes in the following documents - OM High Availability ,Ozone Erasure Coding ,Ozone Snapshot [ozone]

2025-02-04 Thread via GitHub
sreejasahithi opened a new pull request, #7806: URL: https://github.com/apache/ozone/pull/7806 ## What changes were proposed in this pull request? The following documentations have been corrected In OM High Availability document : under OM Bootstrap sub-topic "existing OM’s"

[jira] [Created] (HDDS-12201) Remove unused, dead code in httpfsgateway

2025-02-04 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-12201: --- Summary: Remove unused, dead code in httpfsgateway Key: HDDS-12201 URL: https://issues.apache.org/jira/browse/HDDS-12201 Project: Apache Ozone Issue Ty

[jira] [Created] (HDDS-12200) grammatical changes in the following documents - OM High Availability ,Ozone Erasure Coding ,Ozone Snapshot

2025-02-04 Thread Sreeja (Jira)
Sreeja created HDDS-12200: - Summary: grammatical changes in the following documents - OM High Availability ,Ozone Erasure Coding ,Ozone Snapshot Key: HDDS-12200 URL: https://issues.apache.org/jira/browse/HDDS-12200

[jira] [Created] (HDDS-12199) Explanation in Merge Container RocksDB in DN Documentation is not clear

2025-02-04 Thread Gargi Jaiswal (Jira)
Gargi Jaiswal created HDDS-12199: Summary: Explanation in Merge Container RocksDB in DN Documentation is not clear Key: HDDS-12199 URL: https://issues.apache.org/jira/browse/HDDS-12199 Project: Apache

[jira] [Created] (HDDS-12198) Exclude Recon generated code in coverage

2025-02-04 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-12198: --- Summary: Exclude Recon generated code in coverage Key: HDDS-12198 URL: https://issues.apache.org/jira/browse/HDDS-12198 Project: Apache Ozone Issue Typ

[jira] [Created] (HDDS-12197) Update ozone debug CLI documentation

2025-02-04 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-12197: --- Summary: Update ozone debug CLI documentation Key: HDDS-12197 URL: https://issues.apache.org/jira/browse/HDDS-12197 Project: Apache Ozone Issue Type: S

[jira] [Updated] (HDDS-12196) Document ozone repair cli

2025-02-04 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-12196: Parent: HDDS-11944 Issue Type: Sub-task (was: Task) > Document ozone repair cli >

[jira] [Created] (HDDS-12196) Document ozone repair cli

2025-02-04 Thread Tejaskriya Madhan (Jira)
Tejaskriya Madhan created HDDS-12196: Summary: Document ozone repair cli Key: HDDS-12196 URL: https://issues.apache.org/jira/browse/HDDS-12196 Project: Apache Ozone Issue Type: Task

[jira] [Commented] (HDDS-11839) Mismatch in leadership transfer node ID

2025-02-04 Thread Krishna Kumar Asawa (Jira)
[ https://issues.apache.org/jira/browse/HDDS-11839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17923630#comment-17923630 ] Krishna Kumar Asawa commented on HDDS-11839: [~sshenoy] Adding Sadanand to th

[jira] [Updated] (HDDS-12175) Audit logs in SCM shouldn't print delete txns

2025-02-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12175: -- Labels: pull-request-available (was: ) > Audit logs in SCM shouldn't print delete txns > ---

[PR] HDDS-12175. Audit logs in SCM shouldn't print delete txns [ozone]

2025-02-04 Thread via GitHub
Tejaskriya opened a new pull request, #7805: URL: https://github.com/apache/ozone/pull/7805 ## What changes were proposed in this pull request? As a part of SCM audits for datanode's heartbeats, the command.toString for the commands being sent to the datanodes are logged. For most com

Re: [PR] HDDS-12195. Implement skip() in OzoneFSInputStream [ozone]

2025-02-04 Thread via GitHub
oneonestar commented on PR #7801: URL: https://github.com/apache/ozone/pull/7801#issuecomment-2633223461 Looking into that. Might need sometime to understand the real cause. ![image](https://github.com/user-attachments/assets/ca579cba-d053-46b9-9490-59507fa37dd0) Seems like the Inp

Re: [PR] HDDS-12185. Enhance FileSizeCountTask for Faster Processing. [ozone]

2025-02-04 Thread via GitHub
adoroszlai commented on PR #7796: URL: https://github.com/apache/ozone/pull/7796#issuecomment-2633215059 Please wait for clean CI run in fork before marking "ready for review". Also, please consider opening one PR at a time for related changes. The comment added for #7795 (no throw-a

Re: [PR] HDDS-9377. Implement metrics system with RW-lock instead of synchronization [ozone]

2025-02-04 Thread via GitHub
adoroszlai commented on PR #7708: URL: https://github.com/apache/ozone/pull/7708#issuecomment-2633113616 Thanks @juncevich for the patch. It seems this removes synchronization in some individual metrics, but not in `MetricsSytemImpl` itself. (see also HDDS-12193) My 2 cents: I don'

[PR] Bcsid mismatch repro [ozone]

2025-02-04 Thread via GitHub
swamirishi opened a new pull request, #7804: URL: https://github.com/apache/ozone/pull/7804 ## What changes were proposed in this pull request? Provide a one-liner summary of the changes in the PR **Title** field above. It should be in the form of `HDDS-1234. Short summary of the change

Re: [PR] HDDS-12195. Implement skip() in OzoneFSInputStream [ozone]

2025-02-04 Thread via GitHub
adoroszlai commented on PR #7801: URL: https://github.com/apache/ozone/pull/7801#issuecomment-2633192542 `testOfsHSync` and `testO3fsHSync` seems to fail consistently: ``` TestHSync.testO3fsHSync(boolean)[2] Time elapsed: 0.043 s <<< FAILURE! AssertionFailedError: expected: <32

Re: [PR] HDDS-9377. Implement some metrics with RW-lock instead of synchronization [ozone]

2025-02-04 Thread via GitHub
juncevich commented on PR #7708: URL: https://github.com/apache/ozone/pull/7708#issuecomment-2633192842 > > Could you tell me more about the `better existing metrics library in Ozone`? Is it an existing library in Ozone, or have you offered to change the metrics library from Hadoop to anoth

Re: [PR] HDDS-9377. Implement some metrics with RW-lock instead of synchronization [ozone]

2025-02-04 Thread via GitHub
adoroszlai commented on PR #7708: URL: https://github.com/apache/ozone/pull/7708#issuecomment-2633168885 > Could you tell me more about the `better existing metrics library in Ozone`? Is it an existing library in Ozone, or have you offered to change the metrics library from Hadoop to anothe

[PR] HDDS-12164. Rename DFSConfigKeysLegacy config keys prefixed with 'dfs' [ozone]

2025-02-04 Thread via GitHub
sarvekshayr opened a new pull request, #7803: URL: https://github.com/apache/ozone/pull/7803 ## What changes were proposed in this pull request? The `DFSConfigKeysLegacy` has config keys prefixed with `dfs.` which is the prefix used by HDFS. Instead we should prefix them with `hdds.`, sin

[jira] [Updated] (HDDS-12164) Rename DFSConfigKeysLegacy config keys prefixed with 'dfs'

2025-02-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-12164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-12164: -- Labels: pull-request-available (was: ) > Rename DFSConfigKeysLegacy config keys prefixed with