Re: [PR] [HUDI-9218] Implement HoodieRecordMerger for Flink HoodieRecord [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #13040: URL: https://github.com/apache/hudi/pull/13040#issuecomment-2756932449 ## CI report: * 3dc9e60ea062395f01fb1bd6db192180429fead0 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4467)

Re: [PR] [HUDI-9133] Fallback to legacy schema fetching in case of IllegalAccessError from schema registry client [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #13030: URL: https://github.com/apache/hudi/pull/13030#issuecomment-2756929194 ## CI report: * 4bf3acd9e9a64a2f47672ea99517e795ae06bec3 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4468)

Re: [PR] [HUDI-6895] Change default timeline timezone from local to UTC [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #9794: URL: https://github.com/apache/hudi/pull/9794#issuecomment-2756921612 ## CI report: * f9d87d3de06ffa33eeb9c1fe063907e739885a2d Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4470)

Re: [PR] [HUDI-9205] Refactor Flink tests to avoid sleeping for data results [hudi]

2025-03-26 Thread via GitHub
cshuo commented on PR #13027: URL: https://github.com/apache/hudi/pull/13027#issuecomment-2756852511 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [HUDI-9205] Refactor Flink tests to avoid sleeping for data results [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #13027: URL: https://github.com/apache/hudi/pull/13027#issuecomment-2756839984 ## CI report: * 4b57fa39b3cb33a0a403a05af7a609fced993c82 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4459)

Re: [PR] [HUDI-9218] Implement HoodieRecordMerger for Flink HoodieRecord [hudi]

2025-03-26 Thread via GitHub
danny0405 commented on code in PR #13040: URL: https://github.com/apache/hudi/pull/13040#discussion_r2015761864 ## hudi-client/hudi-flink-client/src/test/java/org/apache/hudi/merge/TestHoodieFlinkRecordMerger.java: ## @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] [HUDI-9218] Implement HoodieRecordMerger for Flink HoodieRecord [hudi]

2025-03-26 Thread via GitHub
danny0405 commented on code in PR #13040: URL: https://github.com/apache/hudi/pull/13040#discussion_r2015759780 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/client/model/OverwriteWithLatestFlinkRecordMerger.java: ## @@ -0,0 +1,50 @@ +/* + * Licensed to the Apac

hudi.git: Error while running github feature from master:.asf.yaml

2025-03-26 Thread Apache Infrastructure
An error occurred while processing the github feature in .asf.yaml: GitHub discussions can only be enabled if a mailing list target exists for it. --- With regards, ASF Infra.

Re: [PR] [HUDI-8635] Support totalLogRecord metric for FG reader based compaction [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #13008: URL: https://github.com/apache/hudi/pull/13008#issuecomment-275688 ## CI report: * 045c5bc7fae64a8ecfb608e32fdfd5aaefa3a2ec Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4475)

Re: [PR] [HUDI-8635] Support totalLogRecord metric for FG reader based compaction [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #13008: URL: https://github.com/apache/hudi/pull/13008#issuecomment-2756877413 ## CI report: * 9f31bc8eb92a06bafdd0771ae5efd342bcf637aa Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4438)

Re: [PR] [HUDI-9205] Refactor Flink tests to avoid sleeping for data results [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #13027: URL: https://github.com/apache/hudi/pull/13027#issuecomment-2756854147 ## CI report: * 4b57fa39b3cb33a0a403a05af7a609fced993c82 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4459) Azu

(hudi) branch master updated: [MINOR] Use configutils to handle defaults for all zk lock provider configs (#13039)

2025-03-26 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new aef7a33fb54 [MINOR] Use configutils to handle d

Re: [PR] [MINOR] Use ConfigUtils to handle defaults for all Zookeeper lock provider configs [hudi]

2025-03-26 Thread via GitHub
danny0405 merged PR #13039: URL: https://github.com/apache/hudi/pull/13039 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apac

Re: [PR] [HUDI-9012] Implement and utilize native writer for HFile [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #12866: URL: https://github.com/apache/hudi/pull/12866#issuecomment-2755663379 ## CI report: * 180b65dd680bf029888057648b10bb4c9f480a97 UNKNOWN * e229e3a1b793a97339602dfc61ce0eae3668ade7 UNKNOWN * 7c9e1fbbb52ae47d26daa0e25fc3d02569c79545 UNKNOWN *

Re: [PR] [MINOR] Improve CI to parallelize tests more and reduce completion time [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #13031: URL: https://github.com/apache/hudi/pull/13031#issuecomment-2755623299 ## CI report: * de52b6c5318439f841188848acfe207c9c53a73e UNKNOWN * 6c240d4c6d39ed9af5687ba7b1772003a208d86c UNKNOWN * dca6d4d9945102e3cf221ce62c62ccbc60c7dcbc Azure: [FAIL

Re: [PR] [HUDI-7902] Cherry-pick partition field format changes from master to 0.x [hudi]

2025-03-26 Thread via GitHub
lokeshj1703 closed pull request #11901: [HUDI-7902] Cherry-pick partition field format changes from master to 0.x URL: https://github.com/apache/hudi/pull/11901 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] [HUDI-9205] Refactor Flink tests to avoid sleeping for data results [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #13027: URL: https://github.com/apache/hudi/pull/13027#issuecomment-2756671391 ## CI report: * 4b57fa39b3cb33a0a403a05af7a609fced993c82 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

Re: [PR] [HUDI-9167] Remove redundant classes in hudi-utilities-slim-bundle [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #12962: URL: https://github.com/apache/hudi/pull/12962#issuecomment-2756197087 ## CI report: * 36d4535992034457adee21957c9bb7196884e61e Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4173)

Re: [PR] [HUDI-9012] Implement and utilize native writer for HFile [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #12866: URL: https://github.com/apache/hudi/pull/12866#issuecomment-2756692084 ## CI report: * 180b65dd680bf029888057648b10bb4c9f480a97 UNKNOWN * e229e3a1b793a97339602dfc61ce0eae3668ade7 UNKNOWN * 7c9e1fbbb52ae47d26daa0e25fc3d02569c79545 UNKNOWN *

hudi.git: Error while running github feature from master:.asf.yaml

2025-03-26 Thread Apache Infrastructure
An error occurred while processing the github feature in .asf.yaml: GitHub discussions can only be enabled if a mailing list target exists for it. --- With regards, ASF Infra.

Re: [PR] [HUDI-9012] Implement and utilize native writer for HFile [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #12866: URL: https://github.com/apache/hudi/pull/12866#issuecomment-2755530013 ## CI report: * 180b65dd680bf029888057648b10bb4c9f480a97 UNKNOWN * e229e3a1b793a97339602dfc61ce0eae3668ade7 UNKNOWN * 7c9e1fbbb52ae47d26daa0e25fc3d02569c79545 UNKNOWN *

Re: [PR] [HUDI-9205] Refactor Flink tests to avoid sleeping for data results [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #13027: URL: https://github.com/apache/hudi/pull/13027#issuecomment-2756673322 ## CI report: * 4b57fa39b3cb33a0a403a05af7a609fced993c82 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4459)

[jira] [Commented] (HUDI-8755) Add more test coverage on delete handling behavior in the file group reader

2025-03-26 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17938788#comment-17938788 ] Lin Liu commented on HUDI-8755: --- [~yihua] , regarding delete, I know we have to do a few thi

(hudi) branch master updated (0185ede3beb -> 550c39b9383)

2025-03-26 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 0185ede3beb [MINOR] Follow up HUDI-8803, optimize vectorized reader by cache 'batchIdxField' (#13023) add 550c39b93

Re: [PR] [HUDI-9088] Fix unnecessary scanning of target table in MERGE INTO on Spark [hudi]

2025-03-26 Thread via GitHub
yihua merged PR #12934: URL: https://github.com/apache/hudi/pull/12934 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.o

(hudi) branch master updated (fb8d27bc4e1 -> 0185ede3beb)

2025-03-26 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from fb8d27bc4e1 [HUDI-9216] Ignore validation of empty databaseName in HoodieTableMetaClient (#13033) add 0185ede3b

Re: [PR] [MINOR]Follow up HUDI-8803, optimize vectorized reader by cache 'batchIdxField' [hudi]

2025-03-26 Thread via GitHub
danny0405 merged PR #13023: URL: https://github.com/apache/hudi/pull/13023 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apac

Re: [PR] [HUDI-9088] Fix unnecessary scanning of target table in MERGE INTO on Spark [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #12934: URL: https://github.com/apache/hudi/pull/12934#issuecomment-2756640514 ## CI report: * 66fc009b65f7b02daa291770212d475392dfd392 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4463)

Re: [PR] [HUDI-9167] Remove redundant classes in hudi-utilities-slim-bundle [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #12962: URL: https://github.com/apache/hudi/pull/12962#issuecomment-2756637560 ## CI report: * 4e88ff75039f58c1842d81ab01804c3166d38d03 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4464)

Re: [PR] [HUDI-9042] Support `TimestampType.DATE_STRING` by `RowDataKeyGen` [hudi]

2025-03-26 Thread via GitHub
danny0405 commented on code in PR #13037: URL: https://github.com/apache/hudi/pull/13037#discussion_r2015171480 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/bulk/RowDataKeyGen.java: ## @@ -271,6 +272,10 @@ private static Object toEpochMilli(Object val,

Re: [PR] [MINOR]Follow up HUDI-8803, optimize vectorized reader by cache 'batchIdxField' [hudi]

2025-03-26 Thread via GitHub
wangyinsheng commented on code in PR #13023: URL: https://github.com/apache/hudi/pull/13023#discussion_r2015554419 ## hudi-spark-datasource/hudi-spark3-common/src/main/java/org/apache/spark/sql/execution/datasources/parquet/Spark3HoodieVectorizedParquetRecordReader.java: ## @@ -

Re: [PR] [HUDI-9088][DNM] Fix unnecessary scanning of target table in MERGE INTO on Spark [hudi]

2025-03-26 Thread via GitHub
yihua closed pull request #12932: [HUDI-9088][DNM] Fix unnecessary scanning of target table in MERGE INTO on Spark URL: https://github.com/apache/hudi/pull/12932 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] [HUDI-6895] Change default timeline timezone from local to UTC [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #9794: URL: https://github.com/apache/hudi/pull/9794#issuecomment-2756597328 ## CI report: * 1936e68fddcb9b7b3b1669bae834b6f0a05b6d95 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4419) *

Re: [PR] [HUDI-6895] Change default timeline timezone from local to UTC [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #9794: URL: https://github.com/apache/hudi/pull/9794#issuecomment-2756594453 ## CI report: * 1936e68fddcb9b7b3b1669bae834b6f0a05b6d95 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4419) *

Re: [PR] [HUDI-9221] Fix tests in TestHoodieBackedMetadata after new default indexes enabled [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #13035: URL: https://github.com/apache/hudi/pull/13035#issuecomment-2756579866 ## CI report: * 6d9e1bf9d011f935b065d8d8c7c86c20aacbd764 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4425)

Re: [PR] [HUDI-9221] Fix tests in TestHoodieBackedMetadata after new default indexes enabled [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #13035: URL: https://github.com/apache/hudi/pull/13035#issuecomment-2756574587 ## CI report: * 6d9e1bf9d011f935b065d8d8c7c86c20aacbd764 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4425)

Re: [PR] [HUDI-9133] Fallback to legacy schema fetching in case of IllegalAccessError from schema registry client [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #13030: URL: https://github.com/apache/hudi/pull/13030#issuecomment-2756574331 ## CI report: * 5936aba2d2178266f60de8d5617b9ab319e33278 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4436)

Re: [PR] [HUDI-9133] Fallback to legacy schema fetching in case of IllegalAccessError from schema registry client [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #13030: URL: https://github.com/apache/hudi/pull/13030#issuecomment-2756569149 ## CI report: * 5936aba2d2178266f60de8d5617b9ab319e33278 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4436)

[jira] [Assigned] (HUDI-8409) Ensure that upgrade and downgrade work with merging configs

2025-03-26 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Liu reassigned HUDI-8409: - Assignee: Y Ethan Guo (was: Lin Liu) > Ensure that upgrade and downgrade work with merging configs > ---

[PR] [MINOR] Use ConfigUtils to handle defaults for all Zookeeper lock provider configs [hudi]

2025-03-26 Thread via GitHub
the-other-tim-brown opened a new pull request, #13039: URL: https://github.com/apache/hudi/pull/13039 ### Change Logs - Use ConfigUtils to avoid null pointers at runtime by using defaults if values are not set. Also eases in any future config renames by factoring in the alternate key

Re: [PR] [HUDI-9218] Implement HoodieRecordMerger for Flink HoodieRecord [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #13040: URL: https://github.com/apache/hudi/pull/13040#issuecomment-2756495747 ## CI report: * 3dc9e60ea062395f01fb1bd6db192180429fead0 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4467)

Re: [PR] [HUDI-9218] Implement HoodieRecordMerger for Flink HoodieRecord [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #13040: URL: https://github.com/apache/hudi/pull/13040#issuecomment-2756491295 ## CI report: * 3dc9e60ea062395f01fb1bd6db192180429fead0 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

[PR] [HUDI-9218] Implement HoodieRecordMerger for Flink HoodieRecord [hudi]

2025-03-26 Thread via GitHub
cshuo opened a new pull request, #13040: URL: https://github.com/apache/hudi/pull/13040 ### Change Logs Implement Flink HoodieRecordMerger for merging HoodieFlinkRecords, which will be then used in some writing paths, e.g, writing COW table and also for Flink FG reader. ### Im

[jira] [Updated] (HUDI-9218) Implement HoodieRecordMerger for Flink HoodieRecord

2025-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-9218: - Labels: pull-request-available (was: ) > Implement HoodieRecordMerger for Flink HoodieRecord > --

[jira] [Updated] (HUDI-9205) Refactor Flink tests to avoid sleeping for data results

2025-03-26 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9205: - Fix Version/s: 1.1.0 > Refactor Flink tests to avoid sleeping for data results > -

Re: [PR] [MINOR] Use ConfigUtils to handle defaults for all Zookeeper lock provider configs [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #13039: URL: https://github.com/apache/hudi/pull/13039#issuecomment-2756447674 ## CI report: * a3f6df888c7b11306f11f7bc89d1f1fbab954371 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4462)

Re: [PR] hudi0.13.1 [hudi]

2025-03-26 Thread via GitHub
hanleiit commented on PR #13021: URL: https://github.com/apache/hudi/pull/13021#issuecomment-2756362118 > @hanleiit Did you apply the new fix then https://github.com/apache/hudi/pull/8935/files? Can you share with use the full error stacktrace? the new fix https://github.com/apache/hudi/

Re: [PR] [HUDI-9216] Ignore validation of empty databaseName in HoodieTableMetaClient [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #13033: URL: https://github.com/apache/hudi/pull/13033#issuecomment-2756083258 ## CI report: * 8ca830a7d0d152875a35a4c73818199700456d42 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4452)

Re: [PR] [HUDI-8635] Support totalLogRecord metric for FG reader based compaction [hudi]

2025-03-26 Thread via GitHub
linliu-code commented on code in PR #13008: URL: https://github.com/apache/hudi/pull/13008#discussion_r2015512474 ## hudi-common/src/test/java/org/apache/hudi/common/table/read/TestHoodieFileGroupReaderBase.java: ## @@ -250,6 +287,42 @@ private Map getCommonConfigs(RecordMergeM

Re: [PR] [HUDI-8635] Support totalLogRecord metric for FG reader based compaction [hudi]

2025-03-26 Thread via GitHub
linliu-code commented on code in PR #13008: URL: https://github.com/apache/hudi/pull/13008#discussion_r2015505409 ## hudi-common/src/test/java/org/apache/hudi/common/table/read/TestHoodieFileGroupReaderBase.java: ## @@ -148,6 +157,34 @@ public void testReadFileGroupInMergeOnRea

Re: [PR] hudi0.13.1 [hudi]

2025-03-26 Thread via GitHub
danny0405 commented on PR #13021: URL: https://github.com/apache/hudi/pull/13021#issuecomment-2756421875 > but why is it not merged into Hudi version 0.13.1, this version of the source code is not modified Because the bug is reported after 0.13.1 is released, we put the fix in 0.14.0

Re: [PR] [HUDI-8635] Support totalLogRecord metric for FG reader based compaction [hudi]

2025-03-26 Thread via GitHub
linliu-code commented on code in PR #13008: URL: https://github.com/apache/hudi/pull/13008#discussion_r2015500768 ## hudi-common/src/main/java/org/apache/hudi/common/table/read/FileGroupRecordBuffer.java: ## @@ -238,6 +243,7 @@ protected Option, Map>> doProcessNextDataRecord(T

Re: [PR] [MINOR]Follow up HUDI-8803, optimize vectorized reader by cache 'batchIdxField' [hudi]

2025-03-26 Thread via GitHub
danny0405 commented on code in PR #13023: URL: https://github.com/apache/hudi/pull/13023#discussion_r2015498057 ## hudi-spark-datasource/hudi-spark3-common/src/main/java/org/apache/spark/sql/execution/datasources/parquet/Spark3HoodieVectorizedParquetRecordReader.java: ## @@ -161

Re: [PR] [WIP][HUDI-8990] Partition Level Bucket Index [hudi]

2025-03-26 Thread via GitHub
danny0405 commented on code in PR #13017: URL: https://github.com/apache/hudi/pull/13017#discussion_r2015181976 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/bucket/HoodieSimpleBucketIndex.java: ## @@ -152,14 +153,18 @@ protected Function> getIndexLocatio

Re: [PR] hudi0.13.1 [hudi]

2025-03-26 Thread via GitHub
hanleiit commented on PR #13021: URL: https://github.com/apache/hudi/pull/13021#issuecomment-2756335127 > @hanleiit Did you apply the new fix then https://github.com/apache/hudi/pull/8935/files? Can you share with use the full error stacktrace? I didn't use the new fix https://github.com

Re: [PR] [MINOR]Follow up HUDI-8803, optimize vectorized reader by cache 'batchIdxField' [hudi]

2025-03-26 Thread via GitHub
wangyinsheng commented on code in PR #13023: URL: https://github.com/apache/hudi/pull/13023#discussion_r2015430063 ## hudi-spark-datasource/hudi-spark3-common/src/main/java/org/apache/spark/sql/execution/datasources/parquet/Spark3HoodieVectorizedParquetRecordReader.java: ## @@ -

Re: [PR] [HUDI-9205] Refactor Flink tests to avoid sleeping for data results [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #13027: URL: https://github.com/apache/hudi/pull/13027#issuecomment-2756300581 ## CI report: * 19f4541c7444e11ea10ea0add62c5008004aa662 UNKNOWN * f422934fdc67925c9a208ba6a6235cd62ca14db9 UNKNOWN * 1fbd532a004b59587b5b7b708c2bcdd6dd48a048 UNKNOWN *

Re: [PR] [DNM] test validating flink bundle [hudi]

2025-03-26 Thread via GitHub
cshuo closed pull request #13011: [DNM] test validating flink bundle URL: https://github.com/apache/hudi/pull/13011 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

Re: [PR] [HUDI-9012] Implement and utilize native writer for HFile [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #12866: URL: https://github.com/apache/hudi/pull/12866#issuecomment-2756274919 ## CI report: * 180b65dd680bf029888057648b10bb4c9f480a97 UNKNOWN * e229e3a1b793a97339602dfc61ce0eae3668ade7 UNKNOWN * 7c9e1fbbb52ae47d26daa0e25fc3d02569c79545 UNKNOWN *

Re: [PR] [MINOR] Use ConfigUtils to handle defaults for all Zookeeper lock provider configs [hudi]

2025-03-26 Thread via GitHub
the-other-tim-brown commented on code in PR #13039: URL: https://github.com/apache/hudi/pull/13039#discussion_r2015309942 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/lock/BaseZookeeperBasedLockProvider.java: ## @@ -69,12 +69,13 @@ public Ba

hudi.git: Error while running github feature from master:.asf.yaml

2025-03-26 Thread Apache Infrastructure
An error occurred while processing the github feature in .asf.yaml: GitHub discussions can only be enabled if a mailing list target exists for it. --- With regards, ASF Infra.

Re: [PR] [HUDI-9167] Remove redundant classes in hudi-utilities-slim-bundle [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #12962: URL: https://github.com/apache/hudi/pull/12962#issuecomment-2756201500 ## CI report: * 7e87a8169d78a53b77101eb34300d664ef2086b7 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4460)

[jira] [Closed] (HUDI-9021) integrate native hfile write with hbase hfile writer

2025-03-26 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Liu closed HUDI-9021. - Resolution: Duplicate > integrate native hfile write with hbase hfile writer > ---

[jira] [Updated] (HUDI-9020) clean all HBase dependency from Hudi

2025-03-26 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Liu updated HUDI-9020: -- Status: Patch Available (was: In Progress) > clean all HBase dependency from Hudi > ---

Re: [PR] [HUDI-9012] Implement and utilize native writer for HFile [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #12866: URL: https://github.com/apache/hudi/pull/12866#issuecomment-2756186301 ## CI report: * 180b65dd680bf029888057648b10bb4c9f480a97 UNKNOWN * e229e3a1b793a97339602dfc61ce0eae3668ade7 UNKNOWN * 7c9e1fbbb52ae47d26daa0e25fc3d02569c79545 UNKNOWN *

Re: [PR] [HUDI-9012] Implement and utilize native writer for HFile [hudi]

2025-03-26 Thread via GitHub
danny0405 commented on PR #12866: URL: https://github.com/apache/hudi/pull/12866#issuecomment-2756185427 I have no more comments on it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[jira] [Updated] (HUDI-9012) Impl and test native HfileWriter

2025-03-26 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Liu updated HUDI-9012: -- Status: Patch Available (was: In Progress) > Impl and test native HfileWriter > ---

[jira] [Updated] (HUDI-8222) Support for Native HFile writer independent of HBase

2025-03-26 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Liu updated HUDI-8222: -- Fix Version/s: 1.1.0 (was: 1.0.2) > Support for Native HFile writer independent of HBase

[jira] [Updated] (HUDI-9021) integrate native hfile write with hbase hfile writer

2025-03-26 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Liu updated HUDI-9021: -- Status: Patch Available (was: In Progress) > integrate native hfile write with hbase hfile writer > ---

[jira] [Commented] (HUDI-9021) integrate native hfile write with hbase hfile writer

2025-03-26 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17938767#comment-17938767 ] Lin Liu commented on HUDI-9021: --- This change of this task is included in https://issues.apa

[jira] [Closed] (HUDI-9044) Performance test

2025-03-26 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Liu closed HUDI-9044. - Resolution: Done > Performance test > > > Key: HUDI-9044 > URL: h

[jira] [Updated] (HUDI-9207) Spark Insert Overwrite Support Row Writer

2025-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-9207: - Labels: pull-request-available (was: ) > Spark Insert Overwrite Support Row Writer >

Re: [PR] [WIP][HUDI-8990] Partition Level Bucket Index [hudi]

2025-03-26 Thread via GitHub
danny0405 commented on code in PR #13017: URL: https://github.com/apache/hudi/pull/13017#discussion_r2015247510 ## hudi-common/src/main/java/org/apache/hudi/common/model/PartitionBucketIndexHashingConfig.java: ## @@ -0,0 +1,342 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] [HUDI-9012] Implement and utilize native writer for HFile [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #12866: URL: https://github.com/apache/hudi/pull/12866#issuecomment-2756167130 ## CI report: * 180b65dd680bf029888057648b10bb4c9f480a97 UNKNOWN * e229e3a1b793a97339602dfc61ce0eae3668ade7 UNKNOWN * 7c9e1fbbb52ae47d26daa0e25fc3d02569c79545 UNKNOWN *

Re: [PR] [WIP][HUDI-8990] Partition Level Bucket Index [hudi]

2025-03-26 Thread via GitHub
danny0405 commented on code in PR #13017: URL: https://github.com/apache/hudi/pull/13017#discussion_r2015245291 ## hudi-common/src/main/java/org/apache/hudi/common/model/PartitionBucketIndexHashingConfig.java: ## @@ -0,0 +1,342 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] [WIP][HUDI-8990] Partition Level Bucket Index [hudi]

2025-03-26 Thread via GitHub
danny0405 commented on code in PR #13017: URL: https://github.com/apache/hudi/pull/13017#discussion_r2015242086 ## hudi-common/src/main/java/org/apache/hudi/common/model/PartitionBucketIndexHashingConfig.java: ## @@ -0,0 +1,342 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] [HUDI-9205] Refactor Flink tests to avoid sleeping for data results [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #13027: URL: https://github.com/apache/hudi/pull/13027#issuecomment-2756119756 ## CI report: * 3be8ae0d190925dbcdab23aca5335e5358de2bfc Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4431)

Re: [PR] [HUDI-9088] Fix unnecessary scanning of target table in MERGE INTO on Spark [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #12934: URL: https://github.com/apache/hudi/pull/12934#issuecomment-2756141374 ## CI report: * 7165c1c40542663e8a1440cbb7a80a62b7e998bb Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4458)

Re: [PR] [MINOR] Use ConfigUtils to handle defaults for all Zookeeper lock provider configs [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #13039: URL: https://github.com/apache/hudi/pull/13039#issuecomment-2756137763 ## CI report: * a3f6df888c7b11306f11f7bc89d1f1fbab954371 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4462)

Re: [PR] [HUDI-9088] Fix unnecessary scanning of target table in MERGE INTO on Spark [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #12934: URL: https://github.com/apache/hudi/pull/12934#issuecomment-2756135630 ## CI report: * 477346e180b098ca131f0b19c5d74075fd33bd1d Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4457)

Re: [PR] [HUDI-9167] Remove redundant classes in hudi-utilities-slim-bundle [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #12962: URL: https://github.com/apache/hudi/pull/12962#issuecomment-2756126984 ## CI report: * 36d4535992034457adee21957c9bb7196884e61e Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4173)

Re: [PR] [MINOR] Use ConfigUtils to handle defaults for all Zookeeper lock provider configs [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #13039: URL: https://github.com/apache/hudi/pull/13039#issuecomment-2756127378 ## CI report: * a3f6df888c7b11306f11f7bc89d1f1fbab954371 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

Re: [PR] [HUDI-9167] Remove redundant classes in hudi-utilities-slim-bundle [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #12962: URL: https://github.com/apache/hudi/pull/12962#issuecomment-2756118909 ## CI report: * 36d4535992034457adee21957c9bb7196884e61e Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4173)

Re: [PR] [WIP][HUDI-8990] Partition Level Bucket Index [hudi]

2025-03-26 Thread via GitHub
danny0405 commented on code in PR #13017: URL: https://github.com/apache/hudi/pull/13017#discussion_r2015186689 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/bucket/partition/PartitionBucketIndexUtils.java: ## @@ -0,0 +1,85 @@ +/* + * Licensed to the Apac

[jira] [Created] (HUDI-9224) Bump Flink version: from 1.19.1 to 1.19.2 and from 1.20.0 to to 1.20.1

2025-03-26 Thread Shuo Cheng (Jira)
Shuo Cheng created HUDI-9224: Summary: Bump Flink version: from 1.19.1 to 1.19.2 and from 1.20.0 to to 1.20.1 Key: HUDI-9224 URL: https://issues.apache.org/jira/browse/HUDI-9224 Project: Apache Hudi

Re: [PR] [WIP][HUDI-8990] Partition Level Bucket Index [hudi]

2025-03-26 Thread via GitHub
danny0405 commented on code in PR #13017: URL: https://github.com/apache/hudi/pull/13017#discussion_r2015187787 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/bucket/partition/RegexRuleEngine.java: ## @@ -0,0 +1,113 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] [WIP][HUDI-8990] Partition Level Bucket Index [hudi]

2025-03-26 Thread via GitHub
danny0405 commented on code in PR #13017: URL: https://github.com/apache/hudi/pull/13017#discussion_r2015191862 ## hudi-client/hudi-client-common/src/test/java/org/apache/hudi/index/bucket/TestPartitionBucketIndexCalculator.java: ## @@ -0,0 +1,142 @@ +/* + * Licensed to the Apac

Re: [PR] [HUDI-9216] Ignore validation of empty databaseName in HoodieTableMetaClient [hudi]

2025-03-26 Thread via GitHub
yihua merged PR #13033: URL: https://github.com/apache/hudi/pull/13033 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.o

Re: [PR] [HUDI-9216] Ignore validation of empty databaseName in HoodieTableMetaClient [hudi]

2025-03-26 Thread via GitHub
yihua commented on PR #13033: URL: https://github.com/apache/hudi/pull/13033#issuecomment-2756086680 Flink IT flakiness is irrelevant. Merging the PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

(hudi) branch master updated (4f63cc06e74 -> fb8d27bc4e1)

2025-03-26 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 4f63cc06e74 [MINOR] Fix flaky test testCreateNewInstantTimes (#13032) add fb8d27bc4e1 [HUDI-9216] Ignore validation

Re: [PR] [HUDI-9205] Refactor Flink tests to avoid sleeping for data results [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #13027: URL: https://github.com/apache/hudi/pull/13027#issuecomment-2756083212 ## CI report: * 3be8ae0d190925dbcdab23aca5335e5358de2bfc Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4431)

Re: [PR] [HUDI-9012] Implement and utilize native writer for HFile [hudi]

2025-03-26 Thread via GitHub
hudi-bot commented on PR #12866: URL: https://github.com/apache/hudi/pull/12866#issuecomment-2756082952 ## CI report: * 180b65dd680bf029888057648b10bb4c9f480a97 UNKNOWN * e229e3a1b793a97339602dfc61ce0eae3668ade7 UNKNOWN * 7c9e1fbbb52ae47d26daa0e25fc3d02569c79545 UNKNOWN *

[jira] [Updated] (HUDI-9088) MIT not doing partition pruning when using partition columns

2025-03-26 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo updated HUDI-9088: -- Status: Patch Available (was: In Progress) > MIT not doing partition pruning when using partition colum

[jira] [Created] (HUDI-9225) Remove support for Flink 1.14

2025-03-26 Thread Shuo Cheng (Jira)
Shuo Cheng created HUDI-9225: Summary: Remove support for Flink 1.14 Key: HUDI-9225 URL: https://issues.apache.org/jira/browse/HUDI-9225 Project: Apache Hudi Issue Type: Improvement R

[jira] [Created] (HUDI-9226) Add support for Flink 2.0

2025-03-26 Thread Shuo Cheng (Jira)
Shuo Cheng created HUDI-9226: Summary: Add support for Flink 2.0 Key: HUDI-9226 URL: https://issues.apache.org/jira/browse/HUDI-9226 Project: Apache Hudi Issue Type: Improvement Repor

[jira] [Commented] (HUDI-8899) Hudi 1.0 backward writer is not able to turn off MDT

2025-03-26 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17937212#comment-17937212 ] sivabalan narayanan commented on HUDI-8899: --- could not reproduce. closing out th

[jira] [Updated] (HUDI-9167) Remove redundant classes in hudi-utilities-slim-bundle

2025-03-26 Thread Y Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y Ethan Guo updated HUDI-9167: -- Status: In Progress (was: Open) > Remove redundant classes in hudi-utilities-slim-bundle >

Re: [PR] [HUDI-9088][DNM] Fix unnecessary scanning of target table in MERGE INTO on Spark [hudi]

2025-03-26 Thread via GitHub
yihua commented on PR #12932: URL: https://github.com/apache/hudi/pull/12932#issuecomment-2756072513 Closing this in favor of #12933 for 0.x releases. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] [WIP][HUDI-8990] Partition Level Bucket Index [hudi]

2025-03-26 Thread via GitHub
danny0405 commented on code in PR #13017: URL: https://github.com/apache/hudi/pull/13017#discussion_r2015176301 ## hudi-common/src/main/java/org/apache/hudi/common/util/hash/BucketIndexUtil.java: ## @@ -35,8 +35,8 @@ public class BucketIndexUtil { * @param parallelism Parall

Re: [PR] Add New blogs related to Apache Hudi [hudi]

2025-03-26 Thread via GitHub
yihua commented on code in PR #13038: URL: https://github.com/apache/hudi/pull/13038#discussion_r2015175293 ## website/blog/2025-03-26-acid-transactions.mdx: ## @@ -0,0 +1,19 @@ +--- +title: "ACID Transactions in an Open Data Lakehouse" +author: Dipankar Mazumdar +category: blog

  1   2   3   >