Re: [PR] [HUDI-9013] Add backwards compatible MDT writer support and reader support with tbl v6 [hudi]

2025-03-17 Thread via GitHub
hudi-bot commented on PR #12948: URL: https://github.com/apache/hudi/pull/12948#issuecomment-2731867025 ## CI report: * 293d1a47c619237651041b9182f414a272f7c5ed Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4239)

Re: [PR] [HUDI-9083] Fixing flakiness with multi writer test [hudi]

2025-03-17 Thread via GitHub
hudi-bot commented on PR #12987: URL: https://github.com/apache/hudi/pull/12987#issuecomment-2731816805 ## CI report: * e4f0db02620d8035cc090a8a0a7cd4cc1b300d97 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4235)

Re: [PR] [HUDI-9083] Fixing flakiness with multi writer test [hudi]

2025-03-17 Thread via GitHub
hudi-bot commented on PR #12987: URL: https://github.com/apache/hudi/pull/12987#issuecomment-2731813363 ## CI report: * e4f0db02620d8035cc090a8a0a7cd4cc1b300d97 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4235)

Re: [PR] [HUDI-7930] Flink Support for Array of Row and Map of Row value [hudi]

2025-03-17 Thread via GitHub
empcl commented on PR #11727: URL: https://github.com/apache/hudi/pull/11727#issuecomment-2731741492 @David-N-Perkins Hello, why do we need to change the written structure? Why do we change the list and element into array structures? Thank you for your answer. ![image](https://github.

Re: [PR] [HUDI-9013] Add backwards compatible MDT writer support and reader support with tbl v6 [hudi]

2025-03-17 Thread via GitHub
lokeshj1703 commented on code in PR #12948: URL: https://github.com/apache/hudi/pull/12948#discussion_r2000240471 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriterTableVersionSix.java: ## @@ -0,0 +1,337 @@ +/* + * Licensed t

Re: [PR] [HUDI-9187] Use thread safe data structures in ExternalSpillableMap [hudi]

2025-03-17 Thread via GitHub
danny0405 commented on code in PR #12982: URL: https://github.com/apache/hudi/pull/12982#discussion_r1999897974 ## hudi-common/src/main/java/org/apache/hudi/common/util/collection/ExternalSpillableMap.java: ## @@ -92,7 +92,7 @@ public class ExternalSpillableMap implements Map k

Re: [PR] [HUDI-9013] Add backwards compatible MDT writer support and reader support with tbl v6 [hudi]

2025-03-17 Thread via GitHub
hudi-bot commented on PR #12948: URL: https://github.com/apache/hudi/pull/12948#issuecomment-2731727203 ## CI report: * 047885b4286dae609122e6573117cfd5dcdca572 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4221)

Re: [PR] [HUDI-9013] Add backwards compatible MDT writer support and reader support with tbl v6 [hudi]

2025-03-17 Thread via GitHub
lokeshj1703 commented on code in PR #12948: URL: https://github.com/apache/hudi/pull/12948#discussion_r2000240010 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestColumnStatsIndexTableVersionSix.scala: ## @@ -0,0 +1,31 @@ +/* + * Licensed to the

Re: [PR] [HUDI-9170] Fixing schema projection with file group reader [hudi]

2025-03-17 Thread via GitHub
hudi-bot commented on PR #12970: URL: https://github.com/apache/hudi/pull/12970#issuecomment-2730853646 ## CI report: * 09b4ba83b5d61cd777c577e483bfe21098725ecc UNKNOWN * b31778b0dd6ccaf619321c6f9b397f7a388c8717 UNKNOWN * 213ea23649287e9843dc659df6803c5c4337f6ba Azure: [FAIL

[jira] [Closed] (HUDI-9044) Performance test

2025-03-17 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Liu closed HUDI-9044. - Resolution: Fixed > Performance test > > > Key: HUDI-9044 > URL:

Re: [PR] [HUDI-9170] Fixing schema projection with file group reader [hudi]

2025-03-17 Thread via GitHub
hudi-bot commented on PR #12970: URL: https://github.com/apache/hudi/pull/12970#issuecomment-2731302112 ## CI report: * 09b4ba83b5d61cd777c577e483bfe21098725ecc UNKNOWN * b31778b0dd6ccaf619321c6f9b397f7a388c8717 UNKNOWN * d8264536a187f6e213ed1eb08d941c0fc86a1e55 UNKNOWN *

[jira] [Commented] (HUDI-9044) Performance test

2025-03-17 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17936352#comment-17936352 ] Lin Liu commented on HUDI-9044: --- Based on the above result, we can claim the native HFileWri

[jira] [Updated] (HUDI-9044) Performance test

2025-03-17 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Liu updated HUDI-9044: -- Status: In Progress (was: Reopened) > Performance test > > > Key: HUDI-9044 >

Re: [I] [SUPPORT] _hoodie_commit_timestamp showing NULL values for some rows when performing a streaming query using Flink SQL [hudi]

2025-03-17 Thread via GitHub
danny0405 commented on issue #12978: URL: https://github.com/apache/hudi/issues/12978#issuecomment-2731468115 I guess we might need to deem this as a metadata field query support from Flink, maybe the Flink metadata column can adapter to it otherwise the schema does not match for the `CREAT

hudi.git: Error while running github feature from master:.asf.yaml

2025-03-17 Thread Apache Infrastructure
An error occurred while processing the github feature in .asf.yaml: GitHub discussions can only be enabled if a mailing list target exists for it. --- With regards, ASF Infra.

(hudi) branch master updated: [HUDI-9170] Fixing schema projection with file group reader (#12970)

2025-03-17 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 1f43b231763 [HUDI-9170] Fixing schema projection wi

Re: [PR] [HUDI-9170] Fixing schema projection with file group reader [hudi]

2025-03-17 Thread via GitHub
yihua merged PR #12970: URL: https://github.com/apache/hudi/pull/12970 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.o

Re: [PR] [HUDI-9170] Fixing schema projection with file group reader [hudi]

2025-03-17 Thread via GitHub
yihua commented on PR #12970: URL: https://github.com/apache/hudi/pull/12970#issuecomment-2731506447 Flink CI failure is irrelevant. Merging this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] [HUDI-9170] Fixing schema projection with file group reader [hudi]

2025-03-17 Thread via GitHub
hudi-bot commented on PR #12970: URL: https://github.com/apache/hudi/pull/12970#issuecomment-2731504121 ## CI report: * 27bdb40132d1be6aa68732a2f147be35a1b03945 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4236)

[jira] [Commented] (HUDI-9044) Performance test

2025-03-17 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17936351#comment-17936351 ] Lin Liu commented on HUDI-9044: --- h3. HBase HFile Writer, Spark 3.5, Hudi 1.1.0-SNAHPSHOT *

[jira] [Updated] (HUDI-7220) Benchmark new HFile reader

2025-03-17 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Liu updated HUDI-7220: -- Status: Open (was: In Progress) > Benchmark new HFile reader > -- > > K

[jira] [Updated] (HUDI-9044) Performance test

2025-03-17 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Liu updated HUDI-9044: -- Status: Patch Available (was: In Progress) > Performance test > > > Key: HUDI-

[jira] [Reopened] (HUDI-9044) Performance test

2025-03-17 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Liu reopened HUDI-9044: --- > Performance test > > > Key: HUDI-9044 > URL: https://issues.apa

Re: [PR] [HUDI-9187] Use thread safe data structures in ExternalSpillableMap [hudi]

2025-03-17 Thread via GitHub
TheR1sing3un commented on code in PR #12982: URL: https://github.com/apache/hudi/pull/12982#discussion_r241532 ## hudi-common/src/main/java/org/apache/hudi/common/util/collection/ExternalSpillableMap.java: ## @@ -92,7 +92,7 @@ public class ExternalSpillableMap implements Ma

Re: [PR] [DNM] Adding support for partial update with compaction [hudi]

2025-03-17 Thread via GitHub
yihua closed pull request #12379: [DNM] Adding support for partial update with compaction URL: https://github.com/apache/hudi/pull/12379 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [HUDI-9187] Use thread safe data structures in ExternalSpillableMap [hudi]

2025-03-17 Thread via GitHub
zhangyue19921010 commented on code in PR #12982: URL: https://github.com/apache/hudi/pull/12982#discussion_r197831 ## hudi-common/src/main/java/org/apache/hudi/common/util/collection/ExternalSpillableMap.java: ## @@ -92,7 +92,7 @@ public class ExternalSpillableMap implement

Re: [PR] [HUDI-9170] Fixing schema projection with file group reader [hudi]

2025-03-17 Thread via GitHub
hudi-bot commented on PR #12970: URL: https://github.com/apache/hudi/pull/12970#issuecomment-2731408354 ## CI report: * 27bdb40132d1be6aa68732a2f147be35a1b03945 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

Re: [PR] [HUDI-9170] Fixing schema projection with file group reader [hudi]

2025-03-17 Thread via GitHub
hudi-bot commented on PR #12970: URL: https://github.com/apache/hudi/pull/12970#issuecomment-2731409898 ## CI report: * 27bdb40132d1be6aa68732a2f147be35a1b03945 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4236)

Re: [PR] [HUDI-9187] Use thread safe data structures in ExternalSpillableMap [hudi]

2025-03-17 Thread via GitHub
the-other-tim-brown commented on code in PR #12982: URL: https://github.com/apache/hudi/pull/12982#discussion_r207943 ## hudi-common/src/main/java/org/apache/hudi/common/util/collection/ExternalSpillableMap.java: ## @@ -92,7 +92,7 @@ public class ExternalSpillableMap implem

[jira] [Updated] (HUDI-7037) Column Stats for Decimal Field From Metadata table is read as Bytes

2025-03-17 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-7037: -- Parent: HUDI-8727 Issue Type: Sub-task (was: Bug) > Column Stats for Decimal Fi

Re: [PR] [HUDI-9144] Flink writer for MOR table supports writing RowData … [hudi]

2025-03-17 Thread via GitHub
danny0405 commented on code in PR #12967: URL: https://github.com/apache/hudi/pull/12967#discussion_r143776 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/io/log/block/HoodieFlinkParquetDataBlock.java: ## @@ -0,0 +1,88 @@ +/* + * Licensed to the Apache Softwa

[jira] [Updated] (HUDI-7507) ongoing concurrent writers with smaller timestamp can cause issues with table services

2025-03-17 Thread Krishen Bhan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishen Bhan updated HUDI-7507: --- Description: *Scenarios:* Although HUDI operations hold a table lock when creating a .requested insta

Re: [PR] [HUDI-9013] Add backwards compatible MDT writer support and reader support with tbl v6 [hudi]

2025-03-17 Thread via GitHub
danny0405 commented on code in PR #12948: URL: https://github.com/apache/hudi/pull/12948#discussion_r113875 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestColumnStatsIndexTableVersionSix.scala: ## @@ -0,0 +1,31 @@ +/* + * Licensed to the Ap

Re: [PR] [HUDI-9144] Flink writer for MOR table supports writing RowData … [hudi]

2025-03-17 Thread via GitHub
danny0405 commented on code in PR #12967: URL: https://github.com/apache/hudi/pull/12967#discussion_r164567 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/io/v2/RowDataLogHandle.java: ## @@ -0,0 +1,324 @@ +/* + * Licensed to the Apache Software Foundation (AS

Re: [PR] [HUDI-9175] Remove the unnecessary MDT metadata check for col_stats index config update [hudi]

2025-03-17 Thread via GitHub
hudi-bot commented on PR #12977: URL: https://github.com/apache/hudi/pull/12977#issuecomment-2730988868 ## CI report: * 85cbd18a47c7093594b0888715a771c6ad423040 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4206)

Re: [PR] [HUDI-9144] Flink writer for MOR table supports writing RowData … [hudi]

2025-03-17 Thread via GitHub
danny0405 commented on code in PR #12967: URL: https://github.com/apache/hudi/pull/12967#discussion_r147626 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/io/storage/row/HoodieRowDataFileWriter.java: ## @@ -50,4 +56,12 @@ public interface HoodieRowDataFileWri

Re: [PR] [HUDI-9144] Flink writer for MOR table supports writing RowData … [hudi]

2025-03-17 Thread via GitHub
danny0405 commented on code in PR #12967: URL: https://github.com/apache/hudi/pull/12967#discussion_r142832 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/client/FlinkRowDataWriteClient.java: ## @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] [HUDI-9144] Flink writer for MOR table supports writing RowData … [hudi]

2025-03-17 Thread via GitHub
danny0405 commented on code in PR #12967: URL: https://github.com/apache/hudi/pull/12967#discussion_r141233 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/io/FlinkWriteHandleFactory.java: ## @@ -283,4 +283,30 @@ public static DeltaCommitWriteHandleFactory g

(hudi-rs) branch dependabot/cargo/chrono-eq-0.4.40 deleted (was cbec8af)

2025-03-17 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/cargo/chrono-eq-0.4.40 in repository https://gitbox.apache.org/repos/asf/hudi-rs.git was cbec8af build(deps): update chrono requirement from =0.4.39 to =0.4.40 The revisions

Re: [PR] build(deps): update chrono requirement from =0.4.39 to =0.4.40 [hudi-rs]

2025-03-17 Thread via GitHub
xushiyan closed pull request #312: build(deps): update chrono requirement from =0.4.39 to =0.4.40 URL: https://github.com/apache/hudi-rs/pull/312 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [HUDI-9187] Use thread safe data structures in ExternalSpillableMap [hudi]

2025-03-17 Thread via GitHub
the-other-tim-brown commented on code in PR #12982: URL: https://github.com/apache/hudi/pull/12982#discussion_r106551 ## hudi-common/src/main/java/org/apache/hudi/common/util/collection/ExternalSpillableMap.java: ## @@ -92,7 +92,7 @@ public class ExternalSpillableMap implem

Re: [PR] [HUDI-9013] Add backwards compatible MDT writer support and reader support with tbl v6 [hudi]

2025-03-17 Thread via GitHub
danny0405 commented on code in PR #12948: URL: https://github.com/apache/hudi/pull/12948#discussion_r116378 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriterTableVersionSix.java: ## @@ -0,0 +1,337 @@ +/* + * Licensed to

Re: [PR] [HUDI-9013] Add backwards compatible MDT writer support and reader support with tbl v6 [hudi]

2025-03-17 Thread via GitHub
danny0405 commented on code in PR #12948: URL: https://github.com/apache/hudi/pull/12948#discussion_r108609 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestColumnStatsIndex.scala: ## @@ -847,7 +856,7 @@ class TestColumnStatsIndex extends Col

Re: [PR] [HUDI-9187] Use thread safe data structures in ExternalSpillableMap [hudi]

2025-03-17 Thread via GitHub
the-other-tim-brown commented on code in PR #12982: URL: https://github.com/apache/hudi/pull/12982#discussion_r105988 ## hudi-common/src/main/java/org/apache/hudi/common/util/collection/ExternalSpillableMap.java: ## @@ -92,7 +92,7 @@ public class ExternalSpillableMap implem

Re: [PR] [HUDI-9013] Add backwards compatible MDT writer support and reader support with tbl v6 [hudi]

2025-03-17 Thread via GitHub
danny0405 commented on code in PR #12948: URL: https://github.com/apache/hudi/pull/12948#discussion_r1998374919 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestColumnStatsIndex.scala: ## @@ -847,7 +856,7 @@ class TestColumnStatsIndex extends Col

Re: [PR] [DOCS] Weekly round of blog additions [hudi]

2025-03-17 Thread via GitHub
yihua merged PR #12980: URL: https://github.com/apache/hudi/pull/12980 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.o

[jira] [Assigned] (HUDI-7037) Column Stats for Decimal Field From Metadata table is read as Bytes

2025-03-17 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-7037: - Assignee: Sagar Sumit > Column Stats for Decimal Field From Metadata table is rea

[jira] [Closed] (HUDI-7331) Test and certify col stats integration with MOR table

2025-03-17 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-7331. - Resolution: Duplicate > Test and certify col stats integration with MOR table > --

[jira] [Commented] (HUDI-7331) Test and certify col stats integration with MOR table

2025-03-17 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17936340#comment-17936340 ] sivabalan narayanan commented on HUDI-7331: --- https://issues.apache.org/jira/brow

Re: [PR] [HUDI-9187] Use thread safe data structures in ExternalSpillableMap [hudi]

2025-03-17 Thread via GitHub
yihua commented on code in PR #12982: URL: https://github.com/apache/hudi/pull/12982#discussion_r1999868607 ## hudi-common/src/main/java/org/apache/hudi/common/util/collection/ExternalSpillableMap.java: ## @@ -92,7 +92,7 @@ public class ExternalSpillableMap implements Map keySi

Re: [PR] [HUDI-9170] Fixing schema projection with file group reader [hudi]

2025-03-17 Thread via GitHub
yihua commented on code in PR #12970: URL: https://github.com/apache/hudi/pull/12970#discussion_r1999757970 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestSparkDataSource.scala: ## @@ -294,6 +299,127 @@ class TestSparkDataSource extends SparkC

Re: [PR] [HUDI-9175] Remove the unnecessary MDT metadata check for col_stats index config update [hudi]

2025-03-17 Thread via GitHub
yihua commented on code in PR #12977: URL: https://github.com/apache/hudi/pull/12977#discussion_r1999707401 ## hudi-common/src/main/java/org/apache/hudi/common/model/ActionType.java: ## @@ -22,5 +22,18 @@ * The supported action types. */ public enum ActionType { - commit,

Re: [PR] [DNM][HUDI-9030] Run Azure CI with table version 6 [hudi]

2025-03-17 Thread via GitHub
yihua commented on PR #12852: URL: https://github.com/apache/hudi/pull/12852#issuecomment-2731214882 Closing the testing-only PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] [DNM][HUDI-9030] Run Azure CI with table version 6 [hudi]

2025-03-17 Thread via GitHub
yihua closed pull request #12852: [DNM][HUDI-9030] Run Azure CI with table version 6 URL: https://github.com/apache/hudi/pull/12852 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] [HUDI-9083] Fixing flakiness with multi writer test [hudi]

2025-03-17 Thread via GitHub
hudi-bot commented on PR #12987: URL: https://github.com/apache/hudi/pull/12987#issuecomment-2731217104 ## CI report: * e4f0db02620d8035cc090a8a0a7cd4cc1b300d97 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4235)

Re: [PR] [HUDI-9170] Fixing schema projection with file group reader [hudi]

2025-03-17 Thread via GitHub
hudi-bot commented on PR #12970: URL: https://github.com/apache/hudi/pull/12970#issuecomment-2731178871 ## CI report: * 09b4ba83b5d61cd777c577e483bfe21098725ecc UNKNOWN * b31778b0dd6ccaf619321c6f9b397f7a388c8717 UNKNOWN * d8264536a187f6e213ed1eb08d941c0fc86a1e55 UNKNOWN *

Re: [PR] partition secondary index only by secondary key [hudi]

2025-03-17 Thread via GitHub
yihua commented on code in PR #12986: URL: https://github.com/apache/hudi/pull/12986#discussion_r1999848876 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java: ## @@ -824,7 +825,8 @@ public Map> getSecondaryIndexRecords(List keys, Stri r

Re: [PR] [DNM] Adding support for partial update with compaction [hudi]

2025-03-17 Thread via GitHub
yihua commented on PR #12379: URL: https://github.com/apache/hudi/pull/12379#issuecomment-2731166700 #12390 is merged to support compaction with partial update through a new merge handle based on the file group reader. Closing this PR. -- This is an automated message from the Apache Git

(hudi) branch asf-site updated: [DOCS] Weekly round of blog additions (#12980)

2025-03-17 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new a06c87d38aa [DOCS] Weekly round of blog additio

[jira] [Updated] (HUDI-7507) ongoing concurrent writers with smaller timestamp can cause issues with table services

2025-03-17 Thread Krishen Bhan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishen Bhan updated HUDI-7507: --- Description: *Scenarios:* Although HUDI operations hold a table lock when creating a .requested insta

Re: [PR] [HUDI-9170] Fixing schema projection with file group reader [hudi]

2025-03-17 Thread via GitHub
hudi-bot commented on PR #12970: URL: https://github.com/apache/hudi/pull/12970#issuecomment-2731142024 ## CI report: * 09b4ba83b5d61cd777c577e483bfe21098725ecc UNKNOWN * b31778b0dd6ccaf619321c6f9b397f7a388c8717 UNKNOWN * d8264536a187f6e213ed1eb08d941c0fc86a1e55 UNKNOWN *

Re: [PR] [HUDI-9175] Remove the unnecessary MDT metadata check for col_stats index config update [hudi]

2025-03-17 Thread via GitHub
yihua commented on PR #12977: URL: https://github.com/apache/hudi/pull/12977#issuecomment-2730988661 Azure CI is green. https://github.com/user-attachments/assets/57814dc5-8e10-47ba-90da-48a3f7ed694f"; /> -- This is an automated message from the Apache Git Service. To respond to th

Re: [PR] [HUDI-9022] Handle records with custom delete markers for MOR [hudi]

2025-03-17 Thread via GitHub
linliu-code commented on code in PR #12843: URL: https://github.com/apache/hudi/pull/12843#discussion_r1999785970 ## hudi-common/src/main/java/org/apache/hudi/common/table/read/PositionBasedFileGroupRecordBuffer.java: ## @@ -61,23 +63,23 @@ * Here the position means that recor

Re: [PR] [HUDI-9083] Fixing flakiness with multi writer test [hudi]

2025-03-17 Thread via GitHub
yihua commented on code in PR #12987: URL: https://github.com/apache/hudi/pull/12987#discussion_r1999724663 ## hudi-spark-datasource/hudi-spark/src/test/java/org/apache/hudi/client/TestHoodieClientMultiWriter.java: ## @@ -834,11 +843,12 @@ public void testMultiWriterWithAsyncTa

Re: [PR] partition secondary index only by secondary key [hudi]

2025-03-17 Thread via GitHub
hudi-bot commented on PR #12986: URL: https://github.com/apache/hudi/pull/12986#issuecomment-2731011416 ## CI report: * 75b3af317b091439efbeb381d09023274a4d1905 UNKNOWN * a57e90cc5b1a32b398cdfaa48e5c296029f27638 UNKNOWN * 8571dbb720e4d23ecf70cb7fe62e60b619355738 Azure: [FAIL

Re: [PR] [HUDI-8840] Support custom delete marker in fg reader [hudi]

2025-03-17 Thread via GitHub
linliu-code closed pull request #12597: [HUDI-8840] Support custom delete marker in fg reader URL: https://github.com/apache/hudi/pull/12597 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] [HUDI-9175] Remove the unnecessary MDT metadata check for col_stats index config update [hudi]

2025-03-17 Thread via GitHub
yihua merged PR #12977: URL: https://github.com/apache/hudi/pull/12977 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.o

Re: [PR] [HUDI-9083] Fixing flakiness with multi writer test [hudi]

2025-03-17 Thread via GitHub
hudi-bot commented on PR #12987: URL: https://github.com/apache/hudi/pull/12987#issuecomment-2731039055 ## CI report: * 244085aaebc4951f33b0bc87084604856983b0b0 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4231)

Re: [PR] [HUDI-8840] Support custom delete marker in fg reader [hudi]

2025-03-17 Thread via GitHub
linliu-code commented on PR #12597: URL: https://github.com/apache/hudi/pull/12597#issuecomment-2731045739 This patch is not needed since the PR for MOR is sufficient for 1.0.2 release. -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [PR] [DOCS] Weekly round of blog additions [hudi]

2025-03-17 Thread via GitHub
yihua commented on PR #12980: URL: https://github.com/apache/hudi/pull/12980#issuecomment-2731028544 I built the changes locally and the new entries show up. Blog: https://github.com/user-attachments/assets/a2ab9e16-6876-412b-85cf-b578281d35f7"; /> Video guide: https://gi

[jira] [Updated] (HUDI-7507) ongoing concurrent writers with smaller timestamp can cause issues with table services

2025-03-17 Thread Krishen Bhan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishen Bhan updated HUDI-7507: --- Description: *Scenarios:* Although HUDI operations hold a table lock when creating a .requested insta

[jira] [Updated] (HUDI-7507) ongoing concurrent writers with smaller timestamp can cause issues with table services

2025-03-17 Thread Krishen Bhan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishen Bhan updated HUDI-7507: --- Attachment: timestmap MOR rollback.png > ongoing concurrent writers with smaller timestamp can cause

Re: [PR] [HUDI-9175] Remove the unnecessary MDT metadata check for col_stats index config update [hudi]

2025-03-17 Thread via GitHub
yihua commented on code in PR #12977: URL: https://github.com/apache/hudi/pull/12977#discussion_r1999705962 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java: ## @@ -1597,10 +1597,9 @@ private static Map getColumnsToIndexWithoutRequiredMetaField

Re: [PR] [HUDI-9127] Fixing completion time generation to honor timezone from table config [hudi]

2025-03-17 Thread via GitHub
yihua merged PR #12926: URL: https://github.com/apache/hudi/pull/12926 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.o

[jira] [Updated] (HUDI-7507) ongoing concurrent writers with smaller timestamp can cause issues with table services

2025-03-17 Thread Krishen Bhan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishen Bhan updated HUDI-7507: --- Attachment: (was: Flowchart (3)-1.png) > ongoing concurrent writers with smaller timestamp can ca

[jira] [Updated] (HUDI-7507) ongoing concurrent writers with smaller timestamp can cause issues with table services

2025-03-17 Thread Krishen Bhan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishen Bhan updated HUDI-7507: --- Description: *Scenarios:* Although HUDI operations hold a table lock when creating a .requested insta

[jira] [Updated] (HUDI-7507) ongoing concurrent writers with smaller timestamp can cause issues with table services

2025-03-17 Thread Krishen Bhan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishen Bhan updated HUDI-7507: --- Attachment: Flowchart (3)-1.png > ongoing concurrent writers with smaller timestamp can cause issues

hudi.git: Error while running github feature from master:.asf.yaml

2025-03-17 Thread Apache Infrastructure
An error occurred while processing the github feature in .asf.yaml: GitHub discussions can only be enabled if a mailing list target exists for it. --- With regards, ASF Infra.

[jira] [Updated] (HUDI-7507) ongoing concurrent writers with smaller timestamp can cause issues with table services

2025-03-17 Thread Krishen Bhan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishen Bhan updated HUDI-7507: --- Description: *Scenarios:* Although HUDI operations hold a table lock when creating a .requested insta

hudi.git: Error while running github feature from master:.asf.yaml

2025-03-17 Thread Apache Infrastructure
An error occurred while processing the github feature in .asf.yaml: GitHub discussions can only be enabled if a mailing list target exists for it. --- With regards, ASF Infra.

(hudi) branch master updated (24fb6f6b3a3 -> c9cf9d5f365)

2025-03-17 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 24fb6f6b3a3 [HUDI-7375] Enable testLogReaderWithDifferentVersionsOfDeleteBlocks (#12668) add c9cf9d5f365 [HUDI-9175

Re: [PR] [HUDI-9175] Remove the unnecessary MDT metadata check for col_stats index config update [hudi]

2025-03-17 Thread via GitHub
hudi-bot commented on PR #12977: URL: https://github.com/apache/hudi/pull/12977#issuecomment-2730986213 ## CI report: * 85cbd18a47c7093594b0888715a771c6ad423040 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

[jira] [Updated] (HUDI-7507) ongoing concurrent writers with smaller timestamp can cause issues with table services

2025-03-17 Thread Krishen Bhan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishen Bhan updated HUDI-7507: --- Attachment: (was: Flowchart (3).png) > ongoing concurrent writers with smaller timestamp can caus

[jira] [Updated] (HUDI-7507) ongoing concurrent writers with smaller timestamp can cause issues with table services

2025-03-17 Thread Krishen Bhan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishen Bhan updated HUDI-7507: --- Attachment: Flowchart (4).png > ongoing concurrent writers with smaller timestamp can cause issues wi

[jira] [Updated] (HUDI-7507) ongoing concurrent writers with smaller timestamp can cause issues with table services

2025-03-17 Thread Krishen Bhan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishen Bhan updated HUDI-7507: --- Description: *Scenarios:* Although HUDI operations hold a table lock when creating a .requested insta

[jira] [Created] (HUDI-9194) Revisit whether to add a test case for HoodieTableMetadataUtil#getColumnsToIndexWithoutRequiredMetaFields

2025-03-17 Thread Y Ethan Guo (Jira)
Y Ethan Guo created HUDI-9194: - Summary: Revisit whether to add a test case for HoodieTableMetadataUtil#getColumnsToIndexWithoutRequiredMetaFields Key: HUDI-9194 URL: https://issues.apache.org/jira/browse/HUDI-9194

Re: [PR] partition secondary index only by secondary key [hudi]

2025-03-17 Thread via GitHub
hudi-bot commented on PR #12986: URL: https://github.com/apache/hudi/pull/12986#issuecomment-2730717221 ## CI report: * dc76da2755095c3b1ff4301942c07ba65b3eb76a Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4227)

[jira] [Updated] (HUDI-7507) ongoing concurrent writers with smaller timestamp can cause issues with table services

2025-03-17 Thread Krishen Bhan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishen Bhan updated HUDI-7507: --- Description: *Scenarios:* Although HUDI operations hold a table lock when creating a .requested insta

[jira] [Updated] (HUDI-7507) ongoing concurrent writers with smaller timestamp can cause issues with table services

2025-03-17 Thread Krishen Bhan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishen Bhan updated HUDI-7507: --- Attachment: Flowchart (3).png > ongoing concurrent writers with smaller timestamp can cause issues wi

Re: [PR] [HUDI-9083] Fixing flakiness with multi writer test [hudi]

2025-03-17 Thread via GitHub
hudi-bot commented on PR #12987: URL: https://github.com/apache/hudi/pull/12987#issuecomment-2730924473 ## CI report: * 244085aaebc4951f33b0bc87084604856983b0b0 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4231)

hudi.git: Error while running github feature from master:.asf.yaml

2025-03-17 Thread Apache Infrastructure
An error occurred while processing the github feature in .asf.yaml: GitHub discussions can only be enabled if a mailing list target exists for it. --- With regards, ASF Infra.

(hudi) branch master updated (5e86be01278 -> 24fb6f6b3a3)

2025-03-17 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 5e86be01278 [HUDI-8345] Delete partition stats index for a partition that is deleted (#12953) add 24fb6f6b3a3 [HUDI

Re: [PR] [HUDI-7375] Enable testLogReaderWithDifferentVersionsOfDeleteBlocks [hudi]

2025-03-17 Thread via GitHub
yihua merged PR #12668: URL: https://github.com/apache/hudi/pull/12668 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.o

hudi.git: Error while running github feature from master:.asf.yaml

2025-03-17 Thread Apache Infrastructure
An error occurred while processing the github feature in .asf.yaml: GitHub discussions can only be enabled if a mailing list target exists for it. --- With regards, ASF Infra.

Re: [PR] [HUDI-9083] Fixing flakiness with multi writer test [hudi]

2025-03-17 Thread via GitHub
hudi-bot commented on PR #12987: URL: https://github.com/apache/hudi/pull/12987#issuecomment-2730939096 ## CI report: * 244085aaebc4951f33b0bc87084604856983b0b0 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4231)

Re: [PR] [HUDI-8345] Delete partition stats index for a partition that is deleted [hudi]

2025-03-17 Thread via GitHub
yihua merged PR #12953: URL: https://github.com/apache/hudi/pull/12953 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.o

Re: [PR] [HUDI-9083] Fixing flakiness with multi writer test [hudi]

2025-03-17 Thread via GitHub
hudi-bot commented on PR #12987: URL: https://github.com/apache/hudi/pull/12987#issuecomment-2730935798 ## CI report: * 244085aaebc4951f33b0bc87084604856983b0b0 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4231)

(hudi) branch master updated (d4cf109cea9 -> 5e86be01278)

2025-03-17 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from d4cf109cea9 [HUDI-9168] Refactor hudi-client-common to not import hudi-aws (#12969) add 5e86be01278 [HUDI-8345] Del

Re: [PR] [HUDI-8345] Delete partition stats index for a partition that is deleted [hudi]

2025-03-17 Thread via GitHub
yihua commented on code in PR #12953: URL: https://github.com/apache/hudi/pull/12953#discussion_r1999677812 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java: ## @@ -2628,11 +2621,42 @@ public static HoodieData convertMetadataToPartitionStatReco

Re: [PR] partition secondary index only by secondary key [hudi]

2025-03-17 Thread via GitHub
hudi-bot commented on PR #12986: URL: https://github.com/apache/hudi/pull/12986#issuecomment-2730867145 ## CI report: * dc76da2755095c3b1ff4301942c07ba65b3eb76a Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4227)

  1   2   >