Re: [PR] [HUDI-9188] Fixing RLI record generation to account for deletes with lower ordering values in MOR log files [hudi]

2025-03-15 Thread via GitHub
hudi-bot commented on PR #12984: URL: https://github.com/apache/hudi/pull/12984#issuecomment-2727219424 ## CI report: * f00a2647cc16d0f0c931c86499b360a69fd9f243 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4214)

Re: [PR] [HUDI-9188] Fixing RLI record generation to account for deletes with lower ordering values in MOR log files [hudi]

2025-03-15 Thread via GitHub
hudi-bot commented on PR #12984: URL: https://github.com/apache/hudi/pull/12984#issuecomment-2727218768 ## CI report: * f00a2647cc16d0f0c931c86499b360a69fd9f243 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4214)

Re: [PR] [HUDI-9188] Fixing RLI record generation to account for deletes with lower ordering values in MOR log files [hudi]

2025-03-15 Thread via GitHub
nsivabalan commented on code in PR #12984: URL: https://github.com/apache/hudi/pull/12984#discussion_r1997495050 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java: ## @@ -839,43 +838,62 @@ public static HoodieData convertMetadataToRecordIndexRec

Re: [PR] [HUDI-9188] Fixing RLI record generation to account for deletes with lower ordering values in MOR log files [hudi]

2025-03-15 Thread via GitHub
hudi-bot commented on PR #12984: URL: https://github.com/apache/hudi/pull/12984#issuecomment-2727214683 ## CI report: * f00a2647cc16d0f0c931c86499b360a69fd9f243 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4214)

Re: [PR] [HUDI-9188] Fixing RLI record generation to account for deletes with lower ordering values in MOR log files [hudi]

2025-03-15 Thread via GitHub
hudi-bot commented on PR #12984: URL: https://github.com/apache/hudi/pull/12984#issuecomment-2727214265 ## CI report: * f00a2647cc16d0f0c931c86499b360a69fd9f243 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

[PR] [HUDI-9188] Fixing RLI record generation to account for deletes with lower ordering values in MOR log files [hudi]

2025-03-15 Thread via GitHub
nsivabalan opened a new pull request, #12984: URL: https://github.com/apache/hudi/pull/12984 ### Change Logs Fixing RLI record generation to account for deletes with lower ordering values in MOR log files. For logs, we were not merging w/ base file to generate RLI records. But there

[jira] [Updated] (HUDI-9188) RLI missing to account for deletes with lower ordering value in MOR log files

2025-03-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-9188: - Labels: pull-request-available (was: ) > RLI missing to account for deletes with lower ordering v

[jira] [Created] (HUDI-9188) RLI missing to account for deletes with lower ordering value in MOR log files

2025-03-15 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-9188: - Summary: RLI missing to account for deletes with lower ordering value in MOR log files Key: HUDI-9188 URL: https://issues.apache.org/jira/browse/HUDI-9188 P

[jira] [Commented] (HUDI-9117) Reduce usage of *LogScanner and standardize on *LogReader..

2025-03-15 Thread Mansi Patel (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17935793#comment-17935793 ] Mansi Patel commented on HUDI-9117: --- Thanks for creating a subtask [~vinoth]. Yes, I've

Re: [PR] feat: support reading avro data block for MOR tables [hudi-rs]

2025-03-15 Thread via GitHub
codecov[bot] commented on PR #309: URL: https://github.com/apache/hudi-rs/pull/309#issuecomment-2727102584 ## [Codecov](https://app.codecov.io/gh/apache/hudi-rs/pull/309?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term

[jira] [Closed] (HUDI-9186) Don't use table state for filegroup reader based parquet file format

2025-03-15 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9186. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 54dcbbb1efd38cf372a8e912e5279cfb3ea19f

[jira] [Updated] (HUDI-9186) Don't use table state for filegroup reader based parquet file format

2025-03-15 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9186: - Fix Version/s: 1.0.2 > Don't use table state for filegroup reader based parquet file format >

Re: [PR] [HUDI-9186] Remove tableState from HoodieFileGroupReaderBasedParquetFileFormat constructor [hudi]

2025-03-15 Thread via GitHub
danny0405 merged PR #12981: URL: https://github.com/apache/hudi/pull/12981 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apac

[PR] feat: support reading avro data block for MOR tables [hudi-rs]

2025-03-15 Thread via GitHub
xushiyan opened a new pull request, #309: URL: https://github.com/apache/hudi-rs/pull/309 ## Description ## How are the changes test-covered - [ ] N/A - [ ] Automated tests (unit and/or integration tests) - [ ] Manual tests - [ ] Details are de

Re: [PR] build(deps): bump ruff from 0.5.2 to 0.9.10 in /python [hudi-rs]

2025-03-15 Thread via GitHub
xushiyan merged PR #307: URL: https://github.com/apache/hudi-rs/pull/307 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

(hudi-rs) branch main updated: build(deps): bump ruff from 0.5.2 to 0.9.10 in /python (#307)

2025-03-15 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/hudi-rs.git The following commit(s) were added to refs/heads/main by this push: new bb56fb8 build(deps): bump ruff from 0.5.2 to 0.9

(hudi-rs) branch dependabot/pip/python/ruff-0.9.10 deleted (was 85dcae5)

2025-03-15 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/pip/python/ruff-0.9.10 in repository https://gitbox.apache.org/repos/asf/hudi-rs.git was 85dcae5 build(deps): bump ruff from 0.5.2 to 0.9.10 in /python The revisions that we

(hudi-rs) branch dependabot/pip/python/ruff-0.9.10 updated (9161e88 -> 85dcae5)

2025-03-15 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/pip/python/ruff-0.9.10 in repository https://gitbox.apache.org/repos/asf/hudi-rs.git discard 9161e88 build(deps): bump ruff from 0.5.2 to 0.9.10 in /python add fb3b798 fix:

Re: [PR] build(deps): bump ruff from 0.5.2 to 0.9.10 in /python [hudi-rs]

2025-03-15 Thread via GitHub
xushiyan commented on PR #307: URL: https://github.com/apache/hudi-rs/pull/307#issuecomment-2726952803 https://github.com/dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

(hudi-rs) branch main updated: fix: make local timezone test more robust (#308)

2025-03-15 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/hudi-rs.git The following commit(s) were added to refs/heads/main by this push: new fb3b798 fix: make local timezone test more robus

Re: [PR] fix: make local timezone test more robust [hudi-rs]

2025-03-15 Thread via GitHub
xushiyan merged PR #308: URL: https://github.com/apache/hudi-rs/pull/308 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

Re: [PR] fix: make local timezone test more robust [hudi-rs]

2025-03-15 Thread via GitHub
codecov[bot] commented on PR #308: URL: https://github.com/apache/hudi-rs/pull/308#issuecomment-2726949045 ## [Codecov](https://app.codecov.io/gh/apache/hudi-rs/pull/308?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term

Re: [PR] build(deps): bump ruff from 0.5.2 to 0.9.10 in /python [hudi-rs]

2025-03-15 Thread via GitHub
dependabot[bot] commented on PR #307: URL: https://github.com/apache/hudi-rs/pull/307#issuecomment-2726948020 Looks like this PR is already up-to-date with main! If you'd still like to recreate it from scratch, overwriting any edits, you can request `@dependabot recreate`. -- This is an

Re: [PR] build(deps): bump ruff from 0.5.2 to 0.9.10 in /python [hudi-rs]

2025-03-15 Thread via GitHub
xushiyan commented on PR #307: URL: https://github.com/apache/hudi-rs/pull/307#issuecomment-2726947996 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] fix: make local timezone test more robust [hudi-rs]

2025-03-15 Thread via GitHub
xushiyan commented on code in PR #308: URL: https://github.com/apache/hudi-rs/pull/308#discussion_r1997244287 ## Cargo.toml: ## @@ -63,7 +63,7 @@ serde_json = { version = "1.0" } # "stdlib" thiserror = { version = "2.0.11" } bytes = { version = "1" } -chrono = { version = "0.

Re: [PR] fix: make local timezone test more robust [hudi-rs]

2025-03-15 Thread via GitHub
xushiyan commented on code in PR #308: URL: https://github.com/apache/hudi-rs/pull/308#discussion_r1997239763 ## Cargo.toml: ## @@ -63,7 +63,7 @@ serde_json = { version = "1.0" } # "stdlib" thiserror = { version = "2.0.11" } bytes = { version = "1" } -chrono = { version = "0.

Re: [PR] fix: make local timezone test more robust [hudi-rs]

2025-03-15 Thread via GitHub
xushiyan commented on code in PR #308: URL: https://github.com/apache/hudi-rs/pull/308#discussion_r1997239763 ## Cargo.toml: ## @@ -63,7 +63,7 @@ serde_json = { version = "1.0" } # "stdlib" thiserror = { version = "2.0.11" } bytes = { version = "1" } -chrono = { version = "0.

(hudi) branch master updated: [HUDI-9123][RFC-91] Claim RFC-91 for storage-based lock provider using conditional writes (#12924)

2025-03-15 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 0f71c31be9c [HUDI-9123][RFC-91] Claim RFC-91 for st

[PR] fix: make local timezone test more robust [hudi-rs]

2025-03-15 Thread via GitHub
xushiyan opened a new pull request, #308: URL: https://github.com/apache/hudi-rs/pull/308 ## Description When DST changes, the test case is affected due to using `Local::now()`. Fixed the timezone to ensure consistent test expectation. ## How are the cha

Re: [PR] feat: iso8601 support in Time-travel & Incremental API [hudi-rs]

2025-03-15 Thread via GitHub
hanbings commented on PR #302: URL: https://github.com/apache/hudi-rs/pull/302#issuecomment-2726871818 It seems that there are some dependency conflicts here (in Github Actions). How should we proceed with the next step? -- This is an automated message from the Apache Git Service. To r

Re: [PR] [HUDI-9166] Introduce schema pruning for delete-record [hudi]

2025-03-15 Thread via GitHub
TheR1sing3un commented on code in PR #12961: URL: https://github.com/apache/hudi/pull/12961#discussion_r1996461612 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/DeleteHoodieTableCommand.scala: ## @@ -41,9 +43,28 @@ case class DeleteHoodieTa

[PR] [HUDI-9120] Fix delete ordering comparison issue [hudi]

2025-03-15 Thread via GitHub
linliu-code opened a new pull request, #12979: URL: https://github.com/apache/hudi/pull/12979 ### Change Logs The root cause: The requestedSchema does not contain the ordering field such that when we merge base file record and log file records, base file cannot find ordering

Re: [PR] [HUDI-8868] Close cachedAllInputFileSlices in BaseHoodieTableFileIndex [hudi]

2025-03-15 Thread via GitHub
yihua commented on code in PR #12878: URL: https://github.com/apache/hudi/pull/12878#discussion_r1996079537 ## hudi-common/src/main/java/org/apache/hudi/BaseHoodieTableFileIndex.java: ## @@ -203,6 +203,9 @@ public int getFileSlicesCount() { @Override public void close()

[jira] [Updated] (HUDI-6694) Fix log file CLI around command blocks

2025-03-15 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo updated HUDI-6694: -- Fix Version/s: 0.14.0 > Fix log file CLI around command blocks > --

[jira] [Updated] (HUDI-8345) Delete partition stats index for a partition that is deleted

2025-03-15 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-8345: -- Priority: Blocker (was: Critical) > Delete partition stats index for a partition that is deleted >

Re: [PR] [HUDI-9152] Improve read/write/compaction performance by reducing the overhead of avro-schema comparison [hudi]

2025-03-15 Thread via GitHub
hudi-bot commented on PR #12949: URL: https://github.com/apache/hudi/pull/12949#issuecomment-2710774018 ## CI report: * 1e8a5b143923475ed2a1f5afc5f8e49fc814464f Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4127)

[jira] [Updated] (HUDI-9124) [Umbrella] RFC-87: Avro elimination in Flink writer

2025-03-15 Thread Mark Bukhner (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Bukhner updated HUDI-9124: --- Summary: [Umbrella] RFC-87: Avro elimination in Flink writer (was: [Umbrella] RFC-87: Avro eliminatio

Re: [PR] [HUDI-9175] Remove the unnecessary MDT metadata check for col_stats index config update [hudi]

2025-03-15 Thread via GitHub
danny0405 commented on code in PR #12977: URL: https://github.com/apache/hudi/pull/12977#discussion_r1996502349 ## hudi-common/src/main/java/org/apache/hudi/common/model/ActionType.java: ## @@ -22,5 +22,18 @@ * The supported action types. */ public enum ActionType { - comm

[jira] [Updated] (HUDI-8289) Get rid of AbstractHoodieLogRecordReader and all children

2025-03-15 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo updated HUDI-8289: -- Parent: HUDI-9117 Issue Type: Sub-task (was: Improvement) > Get rid of AbstractHoodieLogRecordR

Re: [PR] [HUDI-9175] Remove the unnecessary MDT metadata check for col_stats index config update [hudi]

2025-03-15 Thread via GitHub
hudi-bot commented on PR #12977: URL: https://github.com/apache/hudi/pull/12977#issuecomment-2724376052 ## CI report: * 85cbd18a47c7093594b0888715a771c6ad423040 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4206)

[jira] [Updated] (HUDI-8289) Get rid of AbstractHoodieLogRecordScanner and all children

2025-03-15 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo updated HUDI-8289: -- Summary: Get rid of AbstractHoodieLogRecordScanner and all children (was: Get rid of AbstractHoodieLogR

[jira] [Updated] (HUDI-7257) missing datadog configuration metrics on mdt

2025-03-15 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo updated HUDI-7257: -- Fix Version/s: 0.15.0 > missing datadog configuration metrics on mdt > -

[jira] [Assigned] (HUDI-9126) Enable disabled tests(critical ones)

2025-03-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-9126: - Assignee: Y Ethan Guo > Enable disabled tests(critical ones) > --

Re: [PR] [HUDI-7803] Fix bundle validation on Flink 1.18 [hudi]

2025-03-15 Thread via GitHub
hudi-bot commented on PR #12959: URL: https://github.com/apache/hudi/pull/12959#issuecomment-2717190922 ## CI report: * abb1f310b02ac84df17644bb5c36c282f5fa1658 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4167)

[jira] [Updated] (HUDI-9013) Impl: Add backwards compatible MDT writer support and reader support with tbl v6

2025-03-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-9013: -- Labels: hudi-1.0.2 pull-request-available (was: pull-request-available) > Impl: Add bac

Re: [PR] [HUDI-9186] Remove tableState from HoodieFileGroupReaderBasedParquetFileFormat constructor [hudi]

2025-03-15 Thread via GitHub
hudi-bot commented on PR #12981: URL: https://github.com/apache/hudi/pull/12981#issuecomment-2726787206 ## CI report: * c2b3138f615bc7a252ff6abb3246473ad6ec009e Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4209)

[jira] [Assigned] (HUDI-8871) Data issue with hive sync if adding the column in between

2025-03-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-8871: - Assignee: Sagar Sumit > Data issue with hive sync if adding the column in between

[jira] [Updated] (HUDI-8697) Revisit TestHDFSParquetImporter and TestHoodieSnapshotCopier

2025-03-15 Thread Vova Kolmakov (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vova Kolmakov updated HUDI-8697: Status: Open (was: In Progress) > Revisit TestHDFSParquetImporter and TestHoodieSnapshotCopier > --

[jira] [Created] (HUDI-9148) Flink clustering switchs to file group reader

2025-03-15 Thread Shuo Cheng (Jira)
Shuo Cheng created HUDI-9148: Summary: Flink clustering switchs to file group reader Key: HUDI-9148 URL: https://issues.apache.org/jira/browse/HUDI-9148 Project: Apache Hudi Issue Type: Sub-task

[jira] [Updated] (HUDI-9182) Revisit time generation in metadata table

2025-03-15 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo updated HUDI-9182: -- Fix Version/s: 1.1.0 Priority: Blocker (was: Major) > Revisit time generation in metadata tabl

Re: [PR] [HUDI-9152] Improve read/write/compaction performance by reducing the overhead of avro-schema comparison [hudi]

2025-03-15 Thread via GitHub
hudi-bot commented on PR #12949: URL: https://github.com/apache/hudi/pull/12949#issuecomment-2711419790 ## CI report: * 7ea9de982353559b6a806c0fb46a61f23443592d Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4130) Azu

[jira] [Closed] (HUDI-6689) Add record index validation in metadata table validator

2025-03-15 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo closed HUDI-6689. - Resolution: Fixed > Add record index validation in metadata table validator >

[jira] [Updated] (HUDI-6670) Fix timeline check in metadata table validator

2025-03-15 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo updated HUDI-6670: -- Fix Version/s: 0.14.0 > Fix timeline check in metadata table validator > ---

[jira] [Assigned] (HUDI-9117) Reduce usage of *LogScanner and standardize on *LogReader..

2025-03-15 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-9117: Assignee: Vinoth Chandar (was: Mansi Patel) > Reduce usage of *LogScanner and standardize

[jira] [Updated] (HUDI-6664) Fix Java Bulk Insert partitioner for all metadata table partitions

2025-03-15 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo updated HUDI-6664: -- Fix Version/s: 1.0.2 > Fix Java Bulk Insert partitioner for all metadata table partitions >

Re: [PR] [HUDI-9152] Improve read/write/compaction performance by reducing the overhead of avro-schema comparison [hudi]

2025-03-15 Thread via GitHub
hudi-bot commented on PR #12949: URL: https://github.com/apache/hudi/pull/12949#issuecomment-2710875708 ## CI report: * 8aeaf6de309be3e41f6b08096eb96f95a556b7c7 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4128)

[jira] [Updated] (HUDI-9181) Cleaner and safer resource management around hoodieMetadata object

2025-03-15 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo updated HUDI-9181: -- Fix Version/s: 1.1.0 Priority: Blocker (was: Major) > Cleaner and safer resource management ar

[jira] [Comment Edited] (HUDI-8936) MERGE INTO delete does not work in some cases of COMMIT_TIME_ORDERING

2025-03-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17933139#comment-17933139 ] sivabalan narayanan edited comment on HUDI-8936 at 3/6/25 9:47 PM: -

[jira] [Created] (HUDI-9125) Maximum memory for merging do not take effect when compaction using file group reader based mode

2025-03-15 Thread Chaoyang Liu (Jira)
Chaoyang Liu created HUDI-9125: -- Summary: Maximum memory for merging do not take effect when compaction using file group reader based mode Key: HUDI-9125 URL: https://issues.apache.org/jira/browse/HUDI-9125

Re: [PR] [HUDI-9120] Fix delete ordering comparison issue [hudi]

2025-03-15 Thread via GitHub
linliu-code commented on code in PR #12979: URL: https://github.com/apache/hudi/pull/12979#discussion_r1995146243 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/HoodieFileGroupReaderBasedParquetFileFormat.scala: ## @@

Re: [PR] [HUDI-9120] Enable File Group reader by default for table version 6 [hudi]

2025-03-15 Thread via GitHub
linliu-code commented on code in PR #12935: URL: https://github.com/apache/hudi/pull/12935#discussion_r1996120506 ## hudi-common/src/main/java/org/apache/hudi/common/table/read/HoodieBaseFileGroupRecordBuffer.java: ## @@ -424,8 +424,10 @@ protected Option merge(Option older, Map

[PR] [HUDI-9125] Pass compaction/merge related props to HoodieBaseFileGroupRecordBuffer [hudi]

2025-03-15 Thread via GitHub
TheR1sing3un opened a new pull request, #12925: URL: https://github.com/apache/hudi/pull/12925 ### Change Logs 1. pass compaction/merge related props to HoodieBaseFileGroupRecordBuffer _Describe context and summary for this change. Highlight if any code was copied._ ### Impa

Re: [PR] [HUDI-8768] Support bloom filter options when creating expr index using bloom filter [hudi]

2025-03-15 Thread via GitHub
codope commented on code in PR #12919: URL: https://github.com/apache/hudi/pull/12919#discussion_r1983523264 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/feature/index/TestExpressionIndex.scala: ## @@ -1776,7 +1776,7 @@ class TestExpressionIndex e

[jira] [Updated] (HUDI-7267) csi will cause data loss during sql query

2025-03-15 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo updated HUDI-7267: -- Fix Version/s: 1.0.2 > csi will cause data loss during sql query > -

(hudi) branch master updated: [HUDI-8990][RFC-89] Correcting ambiguity in DDL expression (#12968)

2025-03-15 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 95fe5d87f06 [HUDI-8990][RFC-89] Correcting ambiguit

Re: [PR] [HUDI-9175] Remove the unnecessary MDT metadata check for col_stats index config update [hudi]

2025-03-15 Thread via GitHub
danny0405 commented on code in PR #12977: URL: https://github.com/apache/hudi/pull/12977#discussion_r1996523681 ## hudi-common/src/main/java/org/apache/hudi/common/model/ActionType.java: ## @@ -22,5 +22,18 @@ * The supported action types. */ public enum ActionType { - comm

Re: [PR] [HUDI-9170][DNM] Fixing schema projection with file group reader [hudi]

2025-03-15 Thread via GitHub
hudi-bot commented on PR #12970: URL: https://github.com/apache/hudi/pull/12970#issuecomment-2720275957 ## CI report: * 5fee9c21f05426e1f3e34ce9b9bb232324e9575b Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4187)

Re: [PR] [HUDI-9175] Remove the unnecessary MDT metadata check for col_stats index config update [hudi]

2025-03-15 Thread via GitHub
nsivabalan commented on code in PR #12977: URL: https://github.com/apache/hudi/pull/12977#discussion_r1996126118 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java: ## @@ -1597,10 +1597,9 @@ private static Map getColumnsToIndexWithoutRequiredMeta

Re: [PR] [HUDI-9020] Remove hadoop deps from hudi-common [hudi]

2025-03-15 Thread via GitHub
hudi-bot commented on PR #12965: URL: https://github.com/apache/hudi/pull/12965#issuecomment-2719322194 ## CI report: * 16fd5502b0831114ee835287002b01c24b5db88c UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

Re: [PR] [HUDI-9120] Fix delete ordering comparison issue [hudi]

2025-03-15 Thread via GitHub
hudi-bot commented on PR #12979: URL: https://github.com/apache/hudi/pull/12979#issuecomment-2724180904 ## CI report: * 7fbe7ba91d6b07792b558e921ffdcbc156764745 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4202)

[jira] [Updated] (HUDI-7257) missing datadog configuration metrics on mdt

2025-03-15 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo updated HUDI-7257: -- Fix Version/s: 0.14.1 (was: 0.15.0) > missing datadog configuration metrics on md

[jira] [Closed] (HUDI-6648) Allow creation of table with existing files when metadata table is enabled

2025-03-15 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo closed HUDI-6648. - Resolution: Fixed > Allow creation of table with existing files when metadata table is enabled > -

Re: [PR] [HUDI-8768] Support bloom filter options when creating expr index using bloom filter [hudi]

2025-03-15 Thread via GitHub
hudi-bot commented on PR #12919: URL: https://github.com/apache/hudi/pull/12919#issuecomment-2710501271 ## CI report: * 1fadd0bb10f338243630ef81800bcf27c45e9f82 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4122)

[jira] [Updated] (HUDI-8755) Add more test coverage on delete handling behavior in the file group reader

2025-03-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-8755: -- Remaining Estimate: 2h (was: 8h) Original Estimate: 2h (was: 8h) > Add more test

[jira] [Updated] (HUDI-7307) Record index lookup fails for long running Spark jobs on secured Yarn clusters

2025-03-15 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo updated HUDI-7307: -- Fix Version/s: 1.0.2 > Record index lookup fails for long running Spark jobs on secured Yarn clusters >

[jira] [Commented] (HUDI-7259) Multiple tasks get the lock at the same time when use occ

2025-03-15 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17935597#comment-17935597 ] Y Ethan Guo commented on HUDI-7259: --- I don't think the current implementation of `FileSy

[I] [SUPPORT] _hoodie_commit_timestamp showing NULL values for some rows when performing a streaming query using Flink SQL [hudi]

2025-03-15 Thread via GitHub
MaitreyaManohar opened a new issue, #12978: URL: https://github.com/apache/hudi/issues/12978 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? y - Join the mailing list to engage in conversations and get faster support at

[jira] [Updated] (HUDI-7332) The best way to force cleaning hoodie metadata

2025-03-15 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo updated HUDI-7332: -- Fix Version/s: 1.0.2 > The best way to force cleaning hoodie metadata >

Re: [PR] [HUDI-7915] Spark 4 support [hudi]

2025-03-15 Thread via GitHub
hudi-bot commented on PR #12772: URL: https://github.com/apache/hudi/pull/12772#issuecomment-2719460696 ## CI report: * c404e6d16eca90d173c6987e92503d84f344f626 UNKNOWN * df355fc09fee6dffb29e51d11e2ed2419ad5533f Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47

Re: [PR] [MINOR] Remove the unnecessary MDT metadata check for col_stats index config update [hudi]

2025-03-15 Thread via GitHub
hudi-bot commented on PR #12977: URL: https://github.com/apache/hudi/pull/12977#issuecomment-2723108858 ## CI report: * 8d3e8b72baf2c89e4bdb070fa2ea725463cc2270 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4198)

[jira] [Updated] (HUDI-9110) Support Presto and Trino Hudi connector with MDT and data skipping on 1.x release

2025-03-15 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo updated HUDI-9110: -- Status: In Progress (was: Open) > Support Presto and Trino Hudi connector with MDT and data skipping on

[jira] [Updated] (HUDI-7763) Fix that jmx reporter cannot initialized if metadata enables

2025-03-15 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo updated HUDI-7763: -- Fix Version/s: 1.0.0 > Fix that jmx reporter cannot initialized if metadata enables > --

Re: [PR] [DOCS] Weekly round of blog additions [hudi]

2025-03-15 Thread via GitHub
bhasudha commented on PR #12980: URL: https://github.com/apache/hudi/pull/12980#issuecomment-2726014786 @sagarlakshmipathy could you paste the local ui screenshot. The images may not be rendered which is fine -- This is an automated message from the Apache Git Service. To respond to

[jira] [Closed] (HUDI-6620) Support Secondary RLI

2025-03-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-6620. - Resolution: Invalid > Support Secondary RLI > -- > >

[jira] [Updated] (HUDI-6689) Add record index validation in metadata table validator

2025-03-15 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo updated HUDI-6689: -- Fix Version/s: 0.14.0 > Add record index validation in metadata table validator > --

[jira] [Updated] (HUDI-7349) Spark structured streaming didnt work after upgrade from hudi 0.11 to 0.13

2025-03-15 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo updated HUDI-7349: -- Fix Version/s: 1.0.2 > Spark structured streaming didnt work after upgrade from hudi 0.11 to 0.13 >

[jira] [Commented] (HUDI-9117) Reduce usage of *LogScanner and standardize on *LogReader..

2025-03-15 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17935594#comment-17935594 ] Y Ethan Guo commented on HUDI-9117: --- [~vinoth] I pulled in all relevant subtasks we file

Re: [PR] [HUDI-9175] Remove the unnecessary MDT metadata check for col_stats index config update [hudi]

2025-03-15 Thread via GitHub
danny0405 commented on code in PR #12977: URL: https://github.com/apache/hudi/pull/12977#discussion_r1996483658 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java: ## @@ -1597,10 +1597,9 @@ private static Map getColumnsToIndexWithoutRequiredMetaF

[jira] [Assigned] (HUDI-9184) Fix log marker type to APPEND for table version 6 with 1.0 binary

2025-03-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-9184: - Assignee: Lokesh Jain > Fix log marker type to APPEND for table version 6 with 1.

Re: [PR] [HUDI-9186] Remove tableState from HoodieFileGroupReaderBasedParquetFileFormat constructor [hudi]

2025-03-15 Thread via GitHub
hudi-bot commented on PR #12981: URL: https://github.com/apache/hudi/pull/12981#issuecomment-2726746384 ## CI report: * c2b3138f615bc7a252ff6abb3246473ad6ec009e Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4209)

Re: [PR] [HUDI-9186] Remove tableState from HoodieFileGroupReaderBasedParquetFileFormat constructor [hudi]

2025-03-15 Thread via GitHub
hudi-bot commented on PR #12981: URL: https://github.com/apache/hudi/pull/12981#issuecomment-2726745975 ## CI report: * c2b3138f615bc7a252ff6abb3246473ad6ec009e UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

(hudi) branch master updated: [HUDI-9173] Fix issue with inflight compaction and global index lookup (#12976)

2025-03-15 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 34e0a546198 [HUDI-9173] Fix issue with inflight

[jira] [Updated] (HUDI-9173) Writer errors when updating table with global index and compaction inflight

2025-03-15 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9173: - Fix Version/s: 1.0.2 > Writer errors when updating table with global index and compaction inflight > -

[jira] [Closed] (HUDI-9173) Writer errors when updating table with global index and compaction inflight

2025-03-15 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9173. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 34e0a5461988fcf672ed0aee8dc80962713290

[jira] [Assigned] (HUDI-9173) Writer errors when updating table with global index and compaction inflight

2025-03-15 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen reassigned HUDI-9173: Assignee: Timothy Brown > Writer errors when updating table with global index and compaction inflig

Re: [PR] [HUDI-9173] Fix issue with inflight compaction and global index lookup [hudi]

2025-03-15 Thread via GitHub
danny0405 merged PR #12976: URL: https://github.com/apache/hudi/pull/12976 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apac

hudi.git: Error while running github feature from master:.asf.yaml

2025-03-15 Thread Apache Infrastructure
An error occurred while processing the github feature in .asf.yaml: GitHub discussions can only be enabled if a mailing list target exists for it. --- With regards, ASF Infra.

Re: [PR] [HUDI-9173] Fix issue with inflight compaction and global index lookup [hudi]

2025-03-15 Thread via GitHub
danny0405 commented on PR #12976: URL: https://github.com/apache/hudi/pull/12976#issuecomment-2726392438 Azure CI passed: https://dev.azure.com/apachehudi/hudi-oss-ci/_build/results?buildId=4197&view=results -- This is an automated message from the Apache Git Service. To respond to the me