Re: [PR] [HUDI-8775] Expression index on a column should get tracked at partition level if partition stats index is turned on [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12558: URL: https://github.com/apache/hudi/pull/12558#issuecomment-2579389493 ## CI report: * 4b54a2deb80ccce01ec6560927c0d143a8b5c6ba Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2756)

Re: [PR] [HUDI-8841] Fix schema validating exception during flink async cluste… [hudi]

2025-01-09 Thread via GitHub
danny0405 commented on code in PR #12598: URL: https://github.com/apache/hudi/pull/12598#discussion_r1908311946 ## hudi-common/src/main/java/org/apache/hudi/avro/AvroSchemaUtils.java: ## @@ -605,14 +605,16 @@ public static String createSchemaErrorString(String errorMessage, Sch

Re: [PR] [WIP] [HUDI-8796] Silent ignoring of simple bucket index in Flink append mode [hudi]

2025-01-09 Thread via GitHub
geserdugarov commented on code in PR #12545: URL: https://github.com/apache/hudi/pull/12545#discussion_r1908303305 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/utils/Pipelines.java: ## @@ -207,11 +207,30 @@ public static DataStream append( Config

Re: [PR] [HUDI-8800] Introduce SingleSparkConsistentBucketClusteringExecutionStrategy to improve performance [hudi]

2025-01-09 Thread via GitHub
TheR1sing3un commented on code in PR #12537: URL: https://github.com/apache/hudi/pull/12537#discussion_r1908330556 ## hudi-spark-datasource/hudi-spark/src/test/java/org/apache/hudi/functional/TestSparkConsistentBucketClustering.java: ## @@ -110,7 +115,7 @@ public void setup(int

Re: [PR] [WIP] [HUDI-8796] Silent ignoring of simple bucket index in Flink append mode [hudi]

2025-01-09 Thread via GitHub
geserdugarov commented on code in PR #12545: URL: https://github.com/apache/hudi/pull/12545#discussion_r1908303305 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/utils/Pipelines.java: ## @@ -207,11 +207,30 @@ public static DataStream append( Config

Re: [PR] [HUDI-8800] Introduce SingleSparkConsistentBucketClusteringExecutionStrategy to improve performance [hudi]

2025-01-09 Thread via GitHub
TheR1sing3un commented on code in PR #12537: URL: https://github.com/apache/hudi/pull/12537#discussion_r1908331603 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/clustering/run/strategy/SparkJobExecutionStrategy.java: ## @@ -0,0 +1,150 @@ +/* + * Licensed

Re: [PR] [HUDI-8800] Introduce SingleSparkConsistentBucketClusteringExecutionStrategy to improve performance [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12537: URL: https://github.com/apache/hudi/pull/12537#issuecomment-2579417150 ## CI report: * ef470351aa6e521b57e3f3c5e65aa6b9b77f8634 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2755)

Re: [PR] [HUDI-8800] Introduce SingleSparkConsistentBucketClusteringExecutionStrategy to improve performance [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12537: URL: https://github.com/apache/hudi/pull/12537#issuecomment-2579421668 ## CI report: * ef470351aa6e521b57e3f3c5e65aa6b9b77f8634 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2755)

Re: [PR] [HUDI-8775] Expression index on a column should get tracked at partition level if partition stats index is turned on [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12558: URL: https://github.com/apache/hudi/pull/12558#issuecomment-2579441606 ## CI report: * 4b54a2deb80ccce01ec6560927c0d143a8b5c6ba Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2756)

Re: [PR] [HUDI-8775] Expression index on a column should get tracked at partition level if partition stats index is turned on [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12558: URL: https://github.com/apache/hudi/pull/12558#issuecomment-2579444837 ## CI report: * 4b54a2deb80ccce01ec6560927c0d143a8b5c6ba Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2756)

Re: [PR] [HUDI-8775] Expression index on a column should get tracked at partition level if partition stats index is turned on [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12558: URL: https://github.com/apache/hudi/pull/12558#issuecomment-2581900400 ## CI report: * e1e0ea9214a05cd585989e228639f213cc8f033f Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2792)

Re: [PR] [HUDI-8766] Enabling cols stats by default with writer [hudi]

2025-01-09 Thread via GitHub
codope commented on code in PR #12596: URL: https://github.com/apache/hudi/pull/12596#discussion_r1909901588 ## azure-pipelines-20230430.yml: ## @@ -214,7 +214,7 @@ stages: displayName: Top 100 long-running testcases - job: UT_FT_3 displayName: UT sp

Re: [PR] [HUDI-8602] Fix a bug for incremental query [hudi]

2025-01-09 Thread via GitHub
linliu-code commented on code in PR #12385: URL: https://github.com/apache/hudi/pull/12385#discussion_r1909901739 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadIncrementalRelation.scala: ## @@ -209,7 +209,14 @@ trait HoodieIncrementalRelati

Re: [PR] [HUDI-8824] MIT should error out for some assignment clause patterns [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12584: URL: https://github.com/apache/hudi/pull/12584#issuecomment-2581218611 ## CI report: * 631494d4f6e8389bf8c7a7d90a360fc1ea2d159d Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2770)

[jira] [Created] (HUDI-8851) MOR delete query hits NPE when fetching ordering value

2025-01-09 Thread Davis Zhang (Jira)
Davis Zhang created HUDI-8851: - Summary: MOR delete query hits NPE when fetching ordering value Key: HUDI-8851 URL: https://issues.apache.org/jira/browse/HUDI-8851 Project: Apache Hudi Issue Type

[jira] [Assigned] (HUDI-8762) Fix issues around incremental query

2025-01-09 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo reassigned HUDI-8762: - Assignee: Lin Liu (was: Y Ethan Guo) > Fix issues around incremental query > ---

[I] Upgrade pyo3, arrow-rs, datafusion [hudi-rs]

2025-01-09 Thread via GitHub
xushiyan opened a new issue, #242: URL: https://github.com/apache/hudi-rs/issues/242 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: co

Re: [PR] [HUDI-8832] Add merge mode test coverage for DML [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12610: URL: https://github.com/apache/hudi/pull/12610#issuecomment-2581324422 ## CI report: * 4c14c955871ea88e3ff6ccfab667fe434a16a833 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2772)

Re: [PR] [HUDI-8832] Add merge mode test coverage for DML [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12610: URL: https://github.com/apache/hudi/pull/12610#issuecomment-2581326888 ## CI report: * 4c14c955871ea88e3ff6ccfab667fe434a16a833 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2772)

[jira] [Updated] (HUDI-8553) Spark SQL UPDATE and DELETE should write record positions

2025-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-8553: - Labels: pull-request-available (was: ) > Spark SQL UPDATE and DELETE should write record position

[PR] [HUDI-8553] Support writing record positions to log blocks from Spark SQL UPDATE and DELETE statements [hudi]

2025-01-09 Thread via GitHub
yihua opened a new pull request, #12612: URL: https://github.com/apache/hudi/pull/12612 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performance

Re: [PR] [HUDI-8553] Support writing record positions to log blocks from Spark SQL UPDATE and DELETE statements [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12612: URL: https://github.com/apache/hudi/pull/12612#issuecomment-2581581182 ## CI report: * 099eea2fba303c305950fad54010c503aff5c41e UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

Re: [PR] [HUDI-8553] Support writing record positions to log blocks from Spark SQL UPDATE and DELETE statements [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12612: URL: https://github.com/apache/hudi/pull/12612#issuecomment-2581582716 ## CI report: * 099eea2fba303c305950fad54010c503aff5c41e Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2784)

[jira] [Commented] (HUDI-8553) Spark SQL UPDATE and DELETE should write record positions

2025-01-09 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17911738#comment-17911738 ] Y Ethan Guo commented on HUDI-8553: --- I have a draft PR up which makes the prepped upsert

[PR] [HUDI-8624] Avoid check metadata for archived commits in incremental queries [hudi]

2025-01-09 Thread via GitHub
linliu-code opened a new pull request, #12613: URL: https://github.com/apache/hudi/pull/12613 ### Change Logs When start commit is archived, we fall back to full scan. ### Impact Avoid expensive metadata fetching for archived instants. ### Risk level (write none, l

[jira] [Updated] (HUDI-8624) Revisit commitsMetadata fetching from timeline history in MergeOnReadIncrementalRelation

2025-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-8624: - Labels: pull-request-available (was: ) > Revisit commitsMetadata fetching from timeline history i

Re: [PR] [HUDI-8624] Avoid check metadata for archived commits in incremental queries [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12613: URL: https://github.com/apache/hudi/pull/12613#issuecomment-2581592281 ## CI report: * 39ca7fae423367a6f48c5139b257176d22beac02 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

[jira] [Updated] (HUDI-8624) Revisit commitsMetadata fetching from timeline history in MergeOnReadIncrementalRelation

2025-01-09 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Liu updated HUDI-8624: -- Status: Patch Available (was: In Progress) > Revisit commitsMetadata fetching from timeline history in > Merge

[jira] [Updated] (HUDI-8635) Revisit stats generated in HoodieSparkFileGroupReaderBasedMergeHandle

2025-01-09 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo updated HUDI-8635: -- Status: In Progress (was: Open) > Revisit stats generated in HoodieSparkFileGroupReaderBasedMergeHandle

[jira] [Updated] (HUDI-8762) Fix issues around incremental query

2025-01-09 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Liu updated HUDI-8762: -- Status: Patch Available (was: In Progress) > Fix issues around incremental query >

Re: [PR] [HUDI-8775] Expression index on a column should get tracked at partition level if partition stats index is turned on [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12558: URL: https://github.com/apache/hudi/pull/12558#issuecomment-2581813248 ## CI report: * c5912b6788b23621a4dcc609a4d5b4e6ae0af6da Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2767)

Re: [PR] [HUDI-8775] Expression index on a column should get tracked at partition level if partition stats index is turned on [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12558: URL: https://github.com/apache/hudi/pull/12558#issuecomment-2581811657 ## CI report: * c5912b6788b23621a4dcc609a4d5b4e6ae0af6da Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2767)

Re: [PR] [HUDI-8766] Enabling cols stats by default with writer [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12596: URL: https://github.com/apache/hudi/pull/12596#issuecomment-2581822443 ## CI report: * ae2ca606c6cd125f31b7ed029968d0993b1bb0bd UNKNOWN * 71b6a13890909b81c74ce7b138237ab695a08782 UNKNOWN * 15866ae0099c3b58d22329be0e5008b3149cb95f Azure: [FAIL

[jira] [Updated] (HUDI-8796) Silent ignoring of bucket index in Flink append mode

2025-01-09 Thread Geser Dugarov (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Geser Dugarov updated HUDI-8796: Description: Currently, there is no exception when we try to write data in Flink append mode using b

[jira] [Updated] (HUDI-8796) Silent ignoring of bucket index in Flink append mode

2025-01-09 Thread Geser Dugarov (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Geser Dugarov updated HUDI-8796: Summary: Silent ignoring of bucket index in Flink append mode (was: Silent ignoring of simple bucke

(hudi) branch master updated (5f591eec223 -> dc001ea4828)

2025-01-09 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 5f591eec223 [HUDI-8762] Fix a typo in Fix a typo in TestIncrementalQueryWithArchivedInstants (#12611) add dc001ea4

Re: [PR] [HUDI-8775] Expression index on a column should get tracked at partition level if partition stats index is turned on [hudi]

2025-01-09 Thread via GitHub
codope merged PR #12558: URL: https://github.com/apache/hudi/pull/12558 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.

[jira] [Closed] (HUDI-8775) Expression index on a column should get tracked at partition level if partition stats index is turned on

2025-01-09 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-8775. - Resolution: Fixed > Expression index on a column should get tracked at partition level if > partition sta

Re: [PR] [HUDI-8766] Enabling cols stats by default with writer [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12596: URL: https://github.com/apache/hudi/pull/12596#issuecomment-2581888527 ## CI report: * ae2ca606c6cd125f31b7ed029968d0993b1bb0bd UNKNOWN * 71b6a13890909b81c74ce7b138237ab695a08782 UNKNOWN * 15866ae0099c3b58d22329be0e5008b3149cb95f Azure: [FAIL

[jira] [Updated] (HUDI-8839) [Ethan pls check worklog] CDC query: The beforeImageRecords and afterImageRecords are both in-memory hash map, they should be changes to spillable map.

2025-01-09 Thread Davis Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Zhang updated HUDI-8839: -- Summary: [Ethan pls check worklog] CDC query: The beforeImageRecords and  afterImageRecords are both in-m

[jira] [Updated] (HUDI-8839) [Ethan pls check worklog] CDC query: The beforeImageRecords and afterImageRecords are both in-memory hash map, they should be changes to spillable map.

2025-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-8839: - Labels: pull-request-available (was: ) > [Ethan pls check worklog] CDC query: The beforeImageReco

Re: [PR] [HUDI-8839] CdcFileGroupIterator use spillable hashmap [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12592: URL: https://github.com/apache/hudi/pull/12592#issuecomment-2581471756 ## CI report: * 28247026a78dda613a41ed2f039cbf11bb7d5d95 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2779)

[jira] [Assigned] (HUDI-8624) Revisit commitsMetadata fetching from timeline history in MergeOnReadIncrementalRelation

2025-01-09 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Liu reassigned HUDI-8624: - Assignee: Lin Liu > Revisit commitsMetadata fetching from timeline history in > MergeOnReadIncrementalRe

[jira] [Assigned] (HUDI-8172) Make primaryKey and other column configs case insensitive

2025-01-09 Thread Vova Kolmakov (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vova Kolmakov reassigned HUDI-8172: --- Assignee: Vova Kolmakov > Make primaryKey and other column configs case insensitive > ---

[jira] [Updated] (HUDI-8172) Make primaryKey and other column configs case insensitive

2025-01-09 Thread Vova Kolmakov (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vova Kolmakov updated HUDI-8172: Status: In Progress (was: Open) > Make primaryKey and other column configs case insensitive > -

Re: [PR] [HUDI-8796] Silent ignoring of simple bucket index in Flink append mode [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12545: URL: https://github.com/apache/hudi/pull/12545#issuecomment-2581794386 ## CI report: * 3efc78274b41c22ac6d2695e715fd157a9b9a9b8 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2789)

Re: [PR] [HUDI-8800] Introduce SingleSparkConsistentBucketClusteringExecutionStrategy to improve performance [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12537: URL: https://github.com/apache/hudi/pull/12537#issuecomment-2581802070 ## CI report: * 64ad84f40ff6a47df76979a382525fee0cc67d2e Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2790)

Re: [PR] [HUDI-8775] Expression index on a column should get tracked at partition level if partition stats index is turned on [hudi]

2025-01-09 Thread via GitHub
codope commented on code in PR #12558: URL: https://github.com/apache/hudi/pull/12558#discussion_r1909890911 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java: ## @@ -395,7 +392,9 @@ public static Map> convertMetadataToRecords(Hoo if (enabl

[jira] [Commented] (HUDI-8837) Fix reading partition path field on metadata bootstrap table

2025-01-09 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17911710#comment-17911710 ] Y Ethan Guo commented on HUDI-8837: --- The test is added in https://github.com/apache/hudi

[jira] [Commented] (HUDI-8553) Spark SQL UPDATE and DELETE should write record positions

2025-01-09 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17911713#comment-17911713 ] Y Ethan Guo commented on HUDI-8553: --- In the UPDATE and DELETE command, we'll try creatin

[jira] [Updated] (HUDI-8762) Fix issues around incremental query

2025-01-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-8762: - Labels: pull-request-available (was: ) > Fix issues around incremental query > --

[PR] [HUDI-8762] Fix a typo in a test [hudi]

2025-01-09 Thread via GitHub
linliu-code opened a new pull request, #12611: URL: https://github.com/apache/hudi/pull/12611 ### Change Logs The config was not set correctly. ### Impact Fixed a typo. ### Risk level (write none, low medium or high below) None. ### Documentation Upda

Re: [PR] [HUDI-8762] Fix a typo in a test [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12611: URL: https://github.com/apache/hudi/pull/12611#issuecomment-2581543130 ## CI report: * 441dfd77c5036cfac3ce7a84cd7984408f5b6b64 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

Re: [PR] [HUDI-8762] Fix a typo in a test [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12611: URL: https://github.com/apache/hudi/pull/12611#issuecomment-2581544722 ## CI report: * 441dfd77c5036cfac3ce7a84cd7984408f5b6b64 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2783)

Re: [PR] [HUDI-8839] CdcFileGroupIterator use spillable hashmap [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12592: URL: https://github.com/apache/hudi/pull/12592#issuecomment-2581548015 ## CI report: * 423421ec00e72021f081c901ac74891a266b8aa5 UNKNOWN * a5544e3e3d5aa734348b7bfd63820d5b8d98cc33 UNKNOWN * 7ca86e570e17a7db2c7394d62f9d95bda8f439db Azure: [SUCC

Re: [PR] [HUDI-8824] MIT should error out for some assignment clause patterns [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12584: URL: https://github.com/apache/hudi/pull/12584#issuecomment-2581546292 ## CI report: * ffad81180c72f871a9677549e38f1915e5668adb Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2781)

Re: [PR] [HUDI-8796] Silent ignoring of simple bucket index in Flink append mode [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12545: URL: https://github.com/apache/hudi/pull/12545#issuecomment-2581691059 ## CI report: * 20a6a8c042d092026fbed250e5b313e366d2cf61 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2786)

[jira] [Assigned] (HUDI-8854) Support LocalDate with ordering value in DeleteRecord

2025-01-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-8854: - Assignee: sivabalan narayanan > Support LocalDate with ordering value in DeleteRe

[jira] [Created] (HUDI-8854) Support LocalDate with ordering value in DeleteRecord

2025-01-09 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-8854: - Summary: Support LocalDate with ordering value in DeleteRecord Key: HUDI-8854 URL: https://issues.apache.org/jira/browse/HUDI-8854 Project: Apache Hudi

[jira] [Updated] (HUDI-8854) Support LocalDate with ordering value in DeleteRecord

2025-01-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-8854: -- Fix Version/s: 1.0.2 > Support LocalDate with ordering value in DeleteRecord > -

Re: [PR] [HUDI-8800] Introduce SingleSparkConsistentBucketClusteringExecutionStrategy to improve performance [hudi]

2025-01-09 Thread via GitHub
TheR1sing3un commented on code in PR #12537: URL: https://github.com/apache/hudi/pull/12537#discussion_r1909786316 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/cluster/strategy/ClusteringExecutionStrategy.java: ## @@ -67,4 +85,69 @@ protected Hood

Re: [PR] [HUDI-8766] Enabling cols stats by default with writer [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12596: URL: https://github.com/apache/hudi/pull/12596#issuecomment-2581698677 ## CI report: * ae2ca606c6cd125f31b7ed029968d0993b1bb0bd UNKNOWN * 71b6a13890909b81c74ce7b138237ab695a08782 UNKNOWN * a0efb5a7f12042228a5444aeab00f98827dfad3a Azure: [FAIL

Re: [PR] [HUDI-8800] Introduce SingleSparkConsistentBucketClusteringExecutionStrategy to improve performance [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12537: URL: https://github.com/apache/hudi/pull/12537#issuecomment-2581698449 ## CI report: * 6198247de5d01f8edaf4976efffdffa6e6674b64 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2763)

Re: [PR] [HUDI-8800] Introduce SingleSparkConsistentBucketClusteringExecutionStrategy to improve performance [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12537: URL: https://github.com/apache/hudi/pull/12537#issuecomment-2581699957 ## CI report: * 6198247de5d01f8edaf4976efffdffa6e6674b64 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2763)

[jira] [Updated] (HUDI-8851) MOR delete query hits NPE when fetching ordering value

2025-01-09 Thread Davis Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Zhang updated HUDI-8851: -- Description: [https://github.com/apache/hudi/pull/12610] when running the delete statement of the test,

[jira] (HUDI-8624) Revisit commitsMetadata fetching from timeline history in MergeOnReadIncrementalRelation

2025-01-09 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8624 ] Lin Liu deleted comment on HUDI-8624: --- was (Author: JIRAUSER301185): What is the issue here? > Revisit commitsMetadata fetching from timeline history in > MergeOnReadIncrementalRelation > -

[jira] [Updated] (HUDI-8624) Revisit commitsMetadata fetching from timeline history in MergeOnReadIncrementalRelation

2025-01-09 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Liu updated HUDI-8624: -- Status: In Progress (was: Open) > Revisit commitsMetadata fetching from timeline history in > MergeOnReadIncre

Re: [PR] [Hudi-8839] CdcFileGroupIterator use spillable hashmap [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12592: URL: https://github.com/apache/hudi/pull/12592#issuecomment-2581423654 ## CI report: * e720dcfa5656730d01e5f22e5f9a890c08c60e0d Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2738)

Re: [PR] [Hudi-8839] CdcFileGroupIterator use spillable hashmap [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12592: URL: https://github.com/apache/hudi/pull/12592#issuecomment-2581425493 ## CI report: * 28247026a78dda613a41ed2f039cbf11bb7d5d95 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2779)

[jira] [Created] (HUDI-8850) COW DML does not honor Commit

2025-01-09 Thread Davis Zhang (Jira)
Davis Zhang created HUDI-8850: - Summary: COW DML does not honor Commit Key: HUDI-8850 URL: https://issues.apache.org/jira/browse/HUDI-8850 Project: Apache Hudi Issue Type: Bug Reporte

Re: [PR] [HUDI-8828] Test coverage of MIT partial update [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12583: URL: https://github.com/apache/hudi/pull/12583#issuecomment-2581382057 ## CI report: * 5912957233547cef72a3427e482c176537a164b2 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2776)

Re: [PR] [HUDI-8766] Enabling cols stats by default with writer [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12596: URL: https://github.com/apache/hudi/pull/12596#issuecomment-2581382215 ## CI report: * ae2ca606c6cd125f31b7ed029968d0993b1bb0bd UNKNOWN * 71b6a13890909b81c74ce7b138237ab695a08782 UNKNOWN * a0efb5a7f12042228a5444aeab00f98827dfad3a Azure: [FAIL

[jira] [Assigned] (HUDI-8837) Fix reading partition path field on metadata bootstrap table

2025-01-09 Thread Davis Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Zhang reassigned HUDI-8837: - Assignee: Y Ethan Guo (was: Davis Zhang) > Fix reading partition path field on metadata bootstra

[jira] [Commented] (HUDI-8837) Fix reading partition path field on metadata bootstrap table

2025-01-09 Thread Davis Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17911714#comment-17911714 ] Davis Zhang commented on HUDI-8837: --- so we can remove the .drop(partitionColName) in the

Re: [PR] [HUDI-8624] Avoid check metadata for archived commits in incremental queries [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12613: URL: https://github.com/apache/hudi/pull/12613#issuecomment-2581719260 ## CI report: * 8fe93c788b78c9239f8feb90d3d78a90b8153914 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2788)

Re: [PR] [HUDI-8766] Enabling cols stats by default with writer [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12596: URL: https://github.com/apache/hudi/pull/12596#issuecomment-2581720604 ## CI report: * ae2ca606c6cd125f31b7ed029968d0993b1bb0bd UNKNOWN * 71b6a13890909b81c74ce7b138237ab695a08782 UNKNOWN * a0efb5a7f12042228a5444aeab00f98827dfad3a Azure: [FAIL

[jira] [Updated] (HUDI-8762) Fix issues around incremental query

2025-01-09 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo updated HUDI-8762: -- Status: In Progress (was: Open) > Fix issues around incremental query > ---

Re: [PR] [HUDI-8766] Enabling cols stats by default with writer [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12596: URL: https://github.com/apache/hudi/pull/12596#issuecomment-2581279856 ## CI report: * 04faca8ac2311fce83d759a6dbd8efb697ccbb6a Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2773)

[jira] [Updated] (HUDI-8828) merge into partial update on all kinds of table should work [Ethan to check the latest comment on new issues]

2025-01-09 Thread Davis Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Zhang updated HUDI-8828: -- Status: Patch Available (was: In Progress) > merge into partial update on all kinds of table should wor

Re: [PR] [HUDI-8828] Test coverage of MIT partial update [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12583: URL: https://github.com/apache/hudi/pull/12583#issuecomment-2581345336 ## CI report: * fff9de91a5b865e6c07ea9bf9b8672cff90bd243 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2706)

Re: [PR] [HUDI-8828] Test coverage of MIT partial update [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12583: URL: https://github.com/apache/hudi/pull/12583#issuecomment-2581347876 ## CI report: * fff9de91a5b865e6c07ea9bf9b8672cff90bd243 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2706)

[jira] [Commented] (HUDI-8853) Spark sql ALTER TABLE queries are failing on EMR

2025-01-09 Thread Mansi Patel (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17911702#comment-17911702 ] Mansi Patel commented on HUDI-8853: --- ALTER COLUMN is also causing issue. {code:java} spa

Re: [PR] [HUDI-8824] MIT should error out for some assignment clause patterns [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12584: URL: https://github.com/apache/hudi/pull/12584#issuecomment-2581479281 ## CI report: * 631494d4f6e8389bf8c7a7d90a360fc1ea2d159d Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2770)

Re: [PR] [HUDI-8824] MIT should error out for some assignment clause patterns [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12584: URL: https://github.com/apache/hudi/pull/12584#issuecomment-2581481057 ## CI report: * 631494d4f6e8389bf8c7a7d90a360fc1ea2d159d Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2770)

Re: [PR] [HUDI-8839] CdcFileGroupIterator use spillable hashmap [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12592: URL: https://github.com/apache/hudi/pull/12592#issuecomment-2581481123 ## CI report: * 28247026a78dda613a41ed2f039cbf11bb7d5d95 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2779)

[jira] [Assigned] (HUDI-8837) Fix reading partition path field on metadata bootstrap table

2025-01-09 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo reassigned HUDI-8837: - Assignee: Davis Zhang > Fix reading partition path field on metadata bootstrap table > --

[jira] [Updated] (HUDI-8837) Fix reading partition path field on metadata bootstrap table

2025-01-09 Thread Davis Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Zhang updated HUDI-8837: -- Status: In Progress (was: Open) > Fix reading partition path field on metadata bootstrap table > --

Re: [PR] [HUDI-8832] Add merge mode test coverage for DML [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12610: URL: https://github.com/apache/hudi/pull/12610#issuecomment-2581484606 ## CI report: * 5fbd4a15950f9d2b214ce3617164f68ac96fdc4b Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2778)

Re: [PR] [HUDI-8796] Silent ignoring of simple bucket index in Flink append mode [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12545: URL: https://github.com/apache/hudi/pull/12545#issuecomment-2581649185 ## CI report: * 20a6a8c042d092026fbed250e5b313e366d2cf61 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2786)

Re: [PR] [HUDI-8796] Silent ignoring of simple bucket index in Flink append mode [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12545: URL: https://github.com/apache/hudi/pull/12545#issuecomment-2581645223 ## CI report: * 1a9b2ad8ba31a4bfb0c41f65af7d76841a946720 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2576)

Re: [PR] [HUDI-8624] Avoid check metadata for archived commits in incremental queries [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12613: URL: https://github.com/apache/hudi/pull/12613#issuecomment-2581651747 ## CI report: * 39ca7fae423367a6f48c5139b257176d22beac02 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2785)

Re: [PR] [HUDI-8796] Silent ignoring of simple bucket index in Flink append mode [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12545: URL: https://github.com/apache/hudi/pull/12545#issuecomment-2581639192 ## CI report: * 1a9b2ad8ba31a4bfb0c41f65af7d76841a946720 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2576)

Re: [PR] [HUDI-8796] Silent ignoring of simple bucket index in Flink append mode [hudi]

2025-01-09 Thread via GitHub
geserdugarov commented on PR #12545: URL: https://github.com/apache/hudi/pull/12545#issuecomment-2581656971 @zhangyue19921010 , @danny0405 , I've switched fix for bucket index support for append mode to its restriction, due to major problem with only one expected base file for bucket ind

Re: [PR] [HUDI-8766] Enabling cols stats by default with writer [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12596: URL: https://github.com/apache/hudi/pull/12596#issuecomment-2581274438 ## CI report: * da34ecaa061dd1f0bce93c213c43f40b810d Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2748)

Re: [PR] [HUDI-8832] Add merge mode test coverage for DML [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12610: URL: https://github.com/apache/hudi/pull/12610#issuecomment-2581373307 ## CI report: * 4c14c955871ea88e3ff6ccfab667fe434a16a833 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2772)

Re: [PR] [HUDI-8832] Add merge mode test coverage for DML [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12610: URL: https://github.com/apache/hudi/pull/12610#issuecomment-2581375583 ## CI report: * 6142abfcebbf84d3bf32097c7499b60ff11ae0a1 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2774)

Re: [PR] [HUDI-8828] Test coverage of MIT partial update [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12583: URL: https://github.com/apache/hudi/pull/12583#issuecomment-2581379791 ## CI report: * fff9de91a5b865e6c07ea9bf9b8672cff90bd243 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2706)

Re: [PR] [HUDI-8828] Test coverage of MIT partial update [hudi]

2025-01-09 Thread via GitHub
hudi-bot commented on PR #12583: URL: https://github.com/apache/hudi/pull/12583#issuecomment-2581510528 ## CI report: * 757290d3cf1ab9027f2f14f3cd22097f50939a56 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2780)

[jira] [Created] (HUDI-8852) merge into partial update should not need precombine field assignment for partial update

2025-01-09 Thread Davis Zhang (Jira)
Davis Zhang created HUDI-8852: - Summary: merge into partial update should not need precombine field assignment for partial update Key: HUDI-8852 URL: https://issues.apache.org/jira/browse/HUDI-8852 Projec

Re: [PR] [HUDI-8828] Test coverage of MIT partial update [hudi]

2025-01-09 Thread via GitHub
Davis-Zhang-Onehouse commented on code in PR #12583: URL: https://github.com/apache/hudi/pull/12583#discussion_r1909484960 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/dml/TestMergeIntoTable.scala: ## @@ -1336,44 +1339,59 @@ class TestMergeIntoTab

  1   2   3   >