Re: [PR] [HUDI-8881] trigger failure earlier in Flink Hudi sink [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12666: URL: https://github.com/apache/hudi/pull/12666#issuecomment-2599575646 ## CI report: * d04b0f1adb718574c2b2ba5687264c9620d7fed5 UNKNOWN * 268fdd3c756f37d0d3a6debf7f6a138222649ebb UNKNOWN * 5333ece14dc1628d9c069c10b4f26eaa7ad72932 Azure: [FAIL

(hudi-rs) branch main updated: feat: add APIs for MOR snapshot reads (#247)

2025-01-17 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/hudi-rs.git The following commit(s) were added to refs/heads/main by this push: new 3dc5fe2 feat: add APIs for MOR snapshot reads (#

Re: [I] Support MoR snapshot read for parquet log block [hudi-rs]

2025-01-17 Thread via GitHub
xushiyan closed issue #78: Support MoR snapshot read for parquet log block URL: https://github.com/apache/hudi-rs/issues/78 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [PR] feat: add APIs for MOR snapshot reads [hudi-rs]

2025-01-17 Thread via GitHub
xushiyan merged PR #247: URL: https://github.com/apache/hudi-rs/pull/247 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

Re: [I] [SUPPORT]Unable to read new data in streaming mode with specific timestamp [hudi]

2025-01-17 Thread via GitHub
xiearthur commented on issue #12661: URL: https://github.com/apache/hudi/issues/12661#issuecomment-2599537734 Yes, I’m sure. env.setRuntimeMode(RuntimeExecutionMode.STREAMING); This is my current complete write code. ``` public static DataStream readHudiDataStream(St

Re: [PR] fix: include commit_seqno for merge order [hudi-rs]

2025-01-17 Thread via GitHub
xushiyan merged PR #250: URL: https://github.com/apache/hudi-rs/pull/250 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

(hudi-rs) branch main updated: fix: include commit_seqno for merge order (#250)

2025-01-17 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/hudi-rs.git The following commit(s) were added to refs/heads/main by this push: new 6c9f499 fix: include commit_seqno for merge orde

Re: [PR] [HUDI-8881] trigger failure earlier in Flink Hudi sink [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12666: URL: https://github.com/apache/hudi/pull/12666#issuecomment-2599519754 ## CI report: * d04b0f1adb718574c2b2ba5687264c9620d7fed5 UNKNOWN * 268fdd3c756f37d0d3a6debf7f6a138222649ebb UNKNOWN * 2d91da44a581e5ab7063dfecc9d1fe3f424c356d Azure: [SUCC

Re: [PR] fix: include commit_seqno for merge order [hudi-rs]

2025-01-17 Thread via GitHub
codecov[bot] commented on PR #250: URL: https://github.com/apache/hudi-rs/pull/250#issuecomment-2599519355 ## [Codecov](https://app.codecov.io/gh/apache/hudi-rs/pull/250?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term

[PR] fix: include commit_seqno for merge order [hudi-rs]

2025-01-17 Thread via GitHub
xushiyan opened a new pull request, #250: URL: https://github.com/apache/hudi-rs/pull/250 ## Description - Add `MetaField` enum for Hudi meta fields. - Consider `_hoodie_commit_seqno` for merging when precombine values are equal ## How are the changes

Re: [PR] [HUDI-8881] trigger failure earlier in Flink Hudi sink [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12666: URL: https://github.com/apache/hudi/pull/12666#issuecomment-2599517074 ## CI report: * d04b0f1adb718574c2b2ba5687264c9620d7fed5 UNKNOWN * 268fdd3c756f37d0d3a6debf7f6a138222649ebb UNKNOWN * 2d91da44a581e5ab7063dfecc9d1fe3f424c356d Azure: [SUCC

Re: [I] [SUPPORT]Unable to read new data in streaming mode with specific timestamp [hudi]

2025-01-17 Thread via GitHub
xiearthur commented on issue #12661: URL: https://github.com/apache/hudi/issues/12661#issuecomment-2599509180 Yes, we did execute the job in streaming mode by setting: ```java options.put(FlinkOptions.READ_AS_STREAMING.key(), "true"); options.put(FlinkOptions.READ_START_COMMIT.ke

Re: [PR] [HUDI-8883] Add ability to configure BaseHoodieCompactionPlanGenerator using reflection [hudi]

2025-01-17 Thread via GitHub
danny0405 commented on code in PR #12664: URL: https://github.com/apache/hudi/pull/12664#discussion_r1920934628 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/compact/ScheduleCompactionActionExecutor.java: ## @@ -213,4 +216,8 @@ private Long parsedT

Re: [PR] [HUDI-6304] Handle cases when inflight compaction files are removed and job failed leaving pending rollback files [hudi]

2025-01-17 Thread via GitHub
danny0405 merged PR #12644: URL: https://github.com/apache/hudi/pull/12644 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apac

(hudi) branch master updated: [HUDI-6304] Handle cases when inflight compaction files are removed and job failed leaving pending rollback files (#12644)

2025-01-17 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 0178b12b5b5 [HUDI-6304] Handle cases when infli

Re: [PR] [HUDI-8780][RFC-83] Incremental Table Service [hudi]

2025-01-17 Thread via GitHub
danny0405 commented on code in PR #12601: URL: https://github.com/apache/hudi/pull/12601#discussion_r1920932312 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/BaseTableServicePlanActionExecutor.java: ## @@ -116,7 +116,7 @@ public Pair, Set> getIncr

Re: [PR] [HUDI-8163] Refactor UnMergedLogHandler with iterators [hudi]

2025-01-17 Thread via GitHub
danny0405 commented on code in PR #12608: URL: https://github.com/apache/hudi/pull/12608#discussion_r1920931154 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/realtime/RealtimeUnmergedRecordReader.java: ## @@ -105,16 +109,20 @@ private List> getParallelProducers( ) {

Re: [PR] [HUDI-8163] Refactor UnMergedLogHandler with iterators [hudi]

2025-01-17 Thread via GitHub
danny0405 commented on code in PR #12608: URL: https://github.com/apache/hudi/pull/12608#discussion_r1920930956 ## hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieUnMergedLogRecordScanner.java: ## @@ -121,27 +121,43 @@ public interface RecordDeletionCallback {

Re: [PR] [HUDI-8163] Refactor UnMergedLogHandler with iterators [hudi]

2025-01-17 Thread via GitHub
danny0405 commented on code in PR #12608: URL: https://github.com/apache/hudi/pull/12608#discussion_r1920931083 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/FormatUtils.java: ## @@ -237,10 +240,14 @@ private List>> getParallelProducers( )

Re: [I] [SUPPORT]Unable to read new data in streaming mode with specific timestamp [hudi]

2025-01-17 Thread via GitHub
xiearthur commented on issue #12661: URL: https://github.com/apache/hudi/issues/12661#issuecomment-2599472675 I can confirm this issue and have traced the source code to understand why. **Root Cause Analysis:** In `IncrementalInputSplits.java`, the core logic for streaming reads is:

Re: [PR] [MINOR] Bump maven shade plugin version [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12672: URL: https://github.com/apache/hudi/pull/12672#issuecomment-2599470989 ## CI report: * ae94c4f425b453b17a6a42980b59edb32f0ba3bd Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2959)

Re: [PR] [HUDI-8817] fix backward compatibility issue [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12615: URL: https://github.com/apache/hudi/pull/12615#issuecomment-2599457209 ## CI report: * 640ffe558368183f4dc35396f67e2e662f884001 UNKNOWN * a64c849ad2eb0fa59fd87881f16bce8bc812dd36 UNKNOWN * c887fde0567cc83f2c989236353ea5e9749d944c UNKNOWN *

Re: [PR] [HUDI-8297] Get rid of HoodieIdentifier [hudi]

2025-01-17 Thread via GitHub
wombatu-kun merged PR #12663: URL: https://github.com/apache/hudi/pull/12663 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.ap

(hudi) branch master updated: [HUDI-8297] Get rid of HoodieIdentifier (#12663)

2025-01-17 Thread wombatukun
This is an automated email from the ASF dual-hosted git repository. wombatukun pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new f1ef3b4b6ca [HUDI-8297] Get rid of HoodieIdent

Re: [PR] [HUDI-7483] Disable flaky test dimension [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12670: URL: https://github.com/apache/hudi/pull/12670#issuecomment-2599435225 ## CI report: * f5b692d8d48c5bca091f8985cde1483d3c6211af Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2956)

Re: [PR] [MINOR] Bump maven shade plugin version [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12672: URL: https://github.com/apache/hudi/pull/12672#issuecomment-2599425207 ## CI report: * ae94c4f425b453b17a6a42980b59edb32f0ba3bd Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2959)

Re: [PR] [MINOR] Bump maven shade plugin version [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12672: URL: https://github.com/apache/hudi/pull/12672#issuecomment-2599424123 ## CI report: * ae94c4f425b453b17a6a42980b59edb32f0ba3bd UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

[PR] [MINOR] Bump maven shade plugin version [hudi]

2025-01-17 Thread via GitHub
CTTY opened a new pull request, #12672: URL: https://github.com/apache/hudi/pull/12672 ### Change Logs Bump maven shade plugin version from 3.4.0 to 3.5.1. This is necessary to bump JDK version to 20+, as older versioned shade plugin won't support new JDK, sample error: ```

Re: [PR] [HUDI-8680][DNM] Enabling partition stats by default [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12671: URL: https://github.com/apache/hudi/pull/12671#issuecomment-2599416533 ## CI report: * e244110ea34130429b4983cb00f7bfbabef0a816 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2957)

Re: [PR] [HUDI-8072] Enable log file only test for Hive MOR tables [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12669: URL: https://github.com/apache/hudi/pull/12669#issuecomment-2599410331 ## CI report: * dd58c992cd390c6fcf8153f061ba541850286120 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2954)

Re: [PR] [HUDI-8817] fix backward compatibility issue [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12615: URL: https://github.com/apache/hudi/pull/12615#issuecomment-2599381643 ## CI report: * 640ffe558368183f4dc35396f67e2e662f884001 UNKNOWN * a64c849ad2eb0fa59fd87881f16bce8bc812dd36 UNKNOWN * 944075b01aaf135d2547864cbe9dedae080c843f Azure: [FAIL

Re: [PR] [HUDI-7375] Enable testLogReaderWithDifferentVersionsOfDeleteBlocks [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12668: URL: https://github.com/apache/hudi/pull/12668#issuecomment-2599363228 ## CI report: * 6d551c36a59080d872292f2db10069a7b0ba04a0 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2951)

Re: [PR] [HUDI-8817] fix backward compatibility issue [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12615: URL: https://github.com/apache/hudi/pull/12615#issuecomment-2599378597 ## CI report: * 640ffe558368183f4dc35396f67e2e662f884001 UNKNOWN * a64c849ad2eb0fa59fd87881f16bce8bc812dd36 UNKNOWN * 944075b01aaf135d2547864cbe9dedae080c843f Azure: [FAIL

Re: [PR] [HUDI-8817] fix backward compatibility issue [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12615: URL: https://github.com/apache/hudi/pull/12615#issuecomment-2599375364 ## CI report: * 640ffe558368183f4dc35396f67e2e662f884001 UNKNOWN * a64c849ad2eb0fa59fd87881f16bce8bc812dd36 UNKNOWN * 944075b01aaf135d2547864cbe9dedae080c843f Azure: [FAIL

[jira] [Updated] (HUDI-8817) Hudi 1.0.0 cannot create older version table

2025-01-17 Thread Davis Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Zhang updated HUDI-8817: -- Status: Patch Available (was: In Progress) > Hudi 1.0.0 cannot create older version table > ---

Re: [PR] [HUDI-8680][DNM] Enabling partition stats by default [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12671: URL: https://github.com/apache/hudi/pull/12671#issuecomment-2599368533 ## CI report: * 5f059e992edd81800d12495a52f77c5c46a8fb2e Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2955)

Re: [PR] [HUDI-7483] Disable flaky test dimension [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12670: URL: https://github.com/apache/hudi/pull/12670#issuecomment-2599354517 ## CI report: * 56be3f583cabb9761dad40e0e7ccce327b213a8b Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2953)

Re: [PR] [HUDI-8881] trigger failure earlier in Flink Hudi sink [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12666: URL: https://github.com/apache/hudi/pull/12666#issuecomment-2599354474 ## CI report: * d04b0f1adb718574c2b2ba5687264c9620d7fed5 UNKNOWN * 268fdd3c756f37d0d3a6debf7f6a138222649ebb UNKNOWN * 2d91da44a581e5ab7063dfecc9d1fe3f424c356d Azure: [SUCC

Re: [PR] [HUDI-8680][DNM] Enabling partition stats by default [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12671: URL: https://github.com/apache/hudi/pull/12671#issuecomment-2599328627 ## CI report: * 5f059e992edd81800d12495a52f77c5c46a8fb2e Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2955)

Re: [PR] [HUDI-8680][DNM] Enabling partition stats by default [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12671: URL: https://github.com/apache/hudi/pull/12671#issuecomment-2599327229 ## CI report: * 5f059e992edd81800d12495a52f77c5c46a8fb2e Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2955)

Re: [PR] [HUDI-7483] Disable flaky test dimension [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12670: URL: https://github.com/apache/hudi/pull/12670#issuecomment-2599313450 ## CI report: * 56be3f583cabb9761dad40e0e7ccce327b213a8b Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2953)

[jira] [Updated] (HUDI-8817) Hudi 1.0.0 cannot create older version table

2025-01-17 Thread Davis Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Zhang updated HUDI-8817: -- Status: In Progress (was: Open) > Hudi 1.0.0 cannot create older version table > --

Re: [PR] [HUDI-8680][DNM] Enabling partition stats by default [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12671: URL: https://github.com/apache/hudi/pull/12671#issuecomment-2599315253 ## CI report: * 5f059e992edd81800d12495a52f77c5c46a8fb2e Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2955)

Re: [PR] [HUDI-7483] Disable flaky test dimension [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12670: URL: https://github.com/apache/hudi/pull/12670#issuecomment-2599315212 ## CI report: * 56be3f583cabb9761dad40e0e7ccce327b213a8b Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2953)

Re: [PR] [HUDI-8072] Enable log file only test for Hive MOR tables [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12669: URL: https://github.com/apache/hudi/pull/12669#issuecomment-2599313418 ## CI report: * a587795f116a79f51426d35ee18e41716b893195 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2952)

Re: [PR] [HUDI-8680][DNM] Enabling partition stats by default [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12671: URL: https://github.com/apache/hudi/pull/12671#issuecomment-2599313486 ## CI report: * 5f059e992edd81800d12495a52f77c5c46a8fb2e UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

[PR] [HUDI-8680][DNM] Enabling partition stats by default [hudi]

2025-01-17 Thread via GitHub
nsivabalan opened a new pull request, #12671: URL: https://github.com/apache/hudi/pull/12671 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any perform

[jira] [Commented] (HUDI-7353) Fix tests that fails by using memory FSV storage type

2025-01-17 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17914251#comment-17914251 ] Lin Liu commented on HUDI-7353: --- Update:  Tried to enable `TestOrcBootstrap`, and ran.  All

Re: [PR] [HUDI-7483] Disable flaky test dimension [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12670: URL: https://github.com/apache/hudi/pull/12670#issuecomment-2599295816 ## CI report: * 56be3f583cabb9761dad40e0e7ccce327b213a8b Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2953)

Re: [PR] [HUDI-7483] Disable flaky test dimension [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12670: URL: https://github.com/apache/hudi/pull/12670#issuecomment-2599294146 ## CI report: * 56be3f583cabb9761dad40e0e7ccce327b213a8b UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

[jira] [Updated] (HUDI-7483) TestHoodieClientMultiWriter.testMultiWriterWithAsyncTableServicesWithConflict

2025-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7483: - Labels: pull-request-available (was: ) > TestHoodieClientMultiWriter.testMultiWriterWithAsyncTabl

[jira] [Updated] (HUDI-8887) FileSystem lock provider can have more than 1 threads hold the lock at the same time

2025-01-17 Thread Davis Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Zhang updated HUDI-8887: -- Description: related https://issues.apache.org/jira/browse/HUDI-7483 It is clear that when the test err

[jira] [Updated] (HUDI-7483) TestHoodieClientMultiWriter.testMultiWriterWithAsyncTableServicesWithConflict

2025-01-17 Thread Davis Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Zhang updated HUDI-7483: -- Reviewers: Y Ethan Guo (was: sivabalan narayanan) > TestHoodieClientMultiWriter.testMultiWriterWithAsyn

[PR] [HUDI-7483] Disable flaky test dimension [hudi]

2025-01-17 Thread via GitHub
Davis-Zhang-Onehouse opened a new pull request, #12670: URL: https://github.com/apache/hudi/pull/12670 ### Change Logs The filesystem based lock provider might not do atomic operations based on OS/docker container the test uses, which leads to lock provider not functioning correctly

[jira] [Created] (HUDI-8887) FileSystem lock provider can have more than 1 threads hold the lock at the same time

2025-01-17 Thread Davis Zhang (Jira)
Davis Zhang created HUDI-8887: - Summary: FileSystem lock provider can have more than 1 threads hold the lock at the same time Key: HUDI-8887 URL: https://issues.apache.org/jira/browse/HUDI-8887 Project: A

[jira] [Commented] (HUDI-7483) TestHoodieClientMultiWriter.testMultiWriterWithAsyncTableServicesWithConflict

2025-01-17 Thread Davis Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17914248#comment-17914248 ] Davis Zhang commented on HUDI-7483: --- It should be issue with the lock provider we use. F

Re: [PR] repro [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12667: URL: https://github.com/apache/hudi/pull/12667#issuecomment-2599264976 ## CI report: * 9b2807fda1c472c6ca2601c1ac7ec0fa0819e423 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2948)

Re: [PR] [HUDI-8072] Enable log file only test for Hive MOR tables [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12669: URL: https://github.com/apache/hudi/pull/12669#issuecomment-2599255669 ## CI report: * a587795f116a79f51426d35ee18e41716b893195 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2952)

Re: [PR] [HUDI-8072] Enable log file only test for Hive MOR tables [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12669: URL: https://github.com/apache/hudi/pull/12669#issuecomment-2599253667 ## CI report: * a587795f116a79f51426d35ee18e41716b893195 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2952)

Re: [PR] [HUDI-8072] Enable log file only test for Hive MOR tables [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12669: URL: https://github.com/apache/hudi/pull/12669#issuecomment-2599242465 ## CI report: * a587795f116a79f51426d35ee18e41716b893195 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2952)

Re: [PR] [HUDI-8072] Enable log file only test for Hive MOR tables [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12669: URL: https://github.com/apache/hudi/pull/12669#issuecomment-2599240778 ## CI report: * a587795f116a79f51426d35ee18e41716b893195 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

[jira] [Updated] (HUDI-7353) Fix tests that fails by using memory FSV storage type

2025-01-17 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Liu updated HUDI-7353: -- Status: In Progress (was: Open) > Fix tests that fails by using memory FSV storage type > -

[jira] [Updated] (HUDI-8072) Fix Hive FG Reader test testReadLogFilesOnlyInMergeOnReadTable

2025-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-8072: - Labels: pull-request-available (was: ) > Fix Hive FG Reader test testReadLogFilesOnlyInMergeOnRea

[jira] [Updated] (HUDI-8072) Fix Hive FG Reader test testReadLogFilesOnlyInMergeOnReadTable

2025-01-17 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Liu updated HUDI-8072: -- Status: In Progress (was: Open) > Fix Hive FG Reader test testReadLogFilesOnlyInMergeOnReadTable >

[jira] [Updated] (HUDI-8072) Fix Hive FG Reader test testReadLogFilesOnlyInMergeOnReadTable

2025-01-17 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Liu updated HUDI-8072: -- Reviewers: Jonathan Vexler, sivabalan narayanan, Y Ethan Guo > Fix Hive FG Reader test testReadLogFilesOnlyInMer

[jira] [Updated] (HUDI-8072) Fix Hive FG Reader test testReadLogFilesOnlyInMergeOnReadTable

2025-01-17 Thread Lin Liu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Liu updated HUDI-8072: -- Status: Patch Available (was: In Progress) > Fix Hive FG Reader test testReadLogFilesOnlyInMergeOnReadTable > -

[PR] [HUDI-8072] Enable log file only test for Hive MOR tables [hudi]

2025-01-17 Thread via GitHub
linliu-code opened a new pull request, #12669: URL: https://github.com/apache/hudi/pull/12669 ### Change Logs Enable log file only file slice generation and enable the test. ### Impact Fixed a bug in log file file slice generation for Hive, and enabled the corresponding

Re: [PR] [HUDI-7375] Enable testLogReaderWithDifferentVersionsOfDeleteBlocks [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12668: URL: https://github.com/apache/hudi/pull/12668#issuecomment-2599237182 ## CI report: * 6d551c36a59080d872292f2db10069a7b0ba04a0 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2951)

Re: [PR] [HUDI-8881] trigger failure earlier in Flink Hudi sink [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12666: URL: https://github.com/apache/hudi/pull/12666#issuecomment-2599235387 ## CI report: * d04b0f1adb718574c2b2ba5687264c9620d7fed5 UNKNOWN * 89d7c538d2dd185241853473aab04718400c2f27 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-4

Re: [PR] [HUDI-7375] Enable testLogReaderWithDifferentVersionsOfDeleteBlocks [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12668: URL: https://github.com/apache/hudi/pull/12668#issuecomment-2599235468 ## CI report: * 6d551c36a59080d872292f2db10069a7b0ba04a0 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

[PR] [HUDI-7375] Enable a test [hudi]

2025-01-17 Thread via GitHub
linliu-code opened a new pull request, #12668: URL: https://github.com/apache/hudi/pull/12668 ### Change Logs The flakiness has been removed. Enable it. ### Impact More coverage. ### Risk level (write none, low medium or high below) None. ### Document

Re: [PR] [HUDI-8881] trigger failure earlier in Flink Hudi sink [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12666: URL: https://github.com/apache/hudi/pull/12666#issuecomment-2599188479 ## CI report: * d04b0f1adb718574c2b2ba5687264c9620d7fed5 UNKNOWN * 89d7c538d2dd185241853473aab04718400c2f27 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-4

Re: [PR] [MINOR] fix up of HUDI-8804 [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12572: URL: https://github.com/apache/hudi/pull/12572#issuecomment-2599187435 ## CI report: * f86c934a61c117be7c3eb03fc7bbd25217a292bb Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2946)

Re: [PR] [HUDI-8881] trigger failure earlier in Flink Hudi sink [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12666: URL: https://github.com/apache/hudi/pull/12666#issuecomment-2599182917 ## CI report: * 6194d602858d8d1f6fcdb3c53c5ffca7cd71a881 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2947)

Re: [PR] [HUDI-8442] Reduce timeline loads while rolling back failed writes [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12164: URL: https://github.com/apache/hudi/pull/12164#issuecomment-2599126573 ## CI report: * 04421bf0786589b54fc198d46433bd1b78c8818f Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2945)

Re: [PR] [HUDI-8881] trigger failure earlier in Flink Hudi sink [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12666: URL: https://github.com/apache/hudi/pull/12666#issuecomment-2599124746 ## CI report: * 6194d602858d8d1f6fcdb3c53c5ffca7cd71a881 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2947)

Re: [PR] [HUDI-8628] Table Service writer config hard wired to table schema [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12646: URL: https://github.com/apache/hudi/pull/12646#issuecomment-2599122576 ## CI report: * cd1fa812b16dd919c4c7ee7d4260aaeadc2c01be UNKNOWN * 7a7f6943be507571346c0e8f8db23691e0caf4e1 UNKNOWN * 0ff611e506b5bb66aa19f632fcc3c4a0bed18cf0 UNKNOWN *

[jira] [Updated] (HUDI-7483) TestHoodieClientMultiWriter.testMultiWriterWithAsyncTableServicesWithConflict

2025-01-17 Thread Davis Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Zhang updated HUDI-7483: -- Reviewers: sivabalan narayanan > TestHoodieClientMultiWriter.testMultiWriterWithAsyncTableServicesWithCo

Re: [PR] repro [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12667: URL: https://github.com/apache/hudi/pull/12667#issuecomment-2599104378 ## CI report: * 9b2807fda1c472c6ca2601c1ac7ec0fa0819e423 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2948)

[jira] [Assigned] (HUDI-7483) TestHoodieClientMultiWriter.testMultiWriterWithAsyncTableServicesWithConflict

2025-01-17 Thread Davis Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Zhang reassigned HUDI-7483: - Assignee: Davis Zhang > TestHoodieClientMultiWriter.testMultiWriterWithAsyncTableServicesWithConf

Re: [PR] repro [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12667: URL: https://github.com/apache/hudi/pull/12667#issuecomment-2599102280 ## CI report: * 9b2807fda1c472c6ca2601c1ac7ec0fa0819e423 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

[jira] [Commented] (HUDI-7483) TestHoodieClientMultiWriter.testMultiWriterWithAsyncTableServicesWithConflict

2025-01-17 Thread Davis Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17914227#comment-17914227 ] Davis Zhang commented on HUDI-7483: --- PR to repro, check it out and follow intellij confi

[PR] repro [hudi]

2025-01-17 Thread via GitHub
Davis-Zhang-Onehouse opened a new pull request, #12667: URL: https://github.com/apache/hudi/pull/12667 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or a

Re: [PR] [HUDI-8885] Improve the efficiency of initializing the metadata table and table services [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12665: URL: https://github.com/apache/hudi/pull/12665#issuecomment-2599083178 ## CI report: * 65dbf24edfe0f42c0fa5201133b3b1a39f9f746d Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2943)

[jira] [Updated] (HUDI-7483) TestHoodieClientMultiWriter.testMultiWriterWithAsyncTableServicesWithConflict

2025-01-17 Thread Davis Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Zhang updated HUDI-7483: -- Attachment: 7483.zip > TestHoodieClientMultiWriter.testMultiWriterWithAsyncTableServicesWithConflict > -

Re: [PR] [HUDI-8881] trigger failure earlier in Flink Hudi sink [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12666: URL: https://github.com/apache/hudi/pull/12666#issuecomment-2599065101 ## CI report: * 6194d602858d8d1f6fcdb3c53c5ffca7cd71a881 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2947)

Re: [PR] [HUDI-7374] Fix flaky test concurrent compaction test [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12657: URL: https://github.com/apache/hudi/pull/12657#issuecomment-2599039773 ## CI report: * 0f194b964823a627592d3e8e5d65d0af42bb62c8 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2941)

Re: [PR] [HUDI-8881] trigger failure earlier in Flink Hudi sink [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12666: URL: https://github.com/apache/hudi/pull/12666#issuecomment-2599037613 ## CI report: * 6194d602858d8d1f6fcdb3c53c5ffca7cd71a881 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2947)

Re: [PR] [HUDI-8881] trigger failure earlier in Flink Hudi sink [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12666: URL: https://github.com/apache/hudi/pull/12666#issuecomment-2599035416 ## CI report: * 6194d602858d8d1f6fcdb3c53c5ffca7cd71a881 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2947)

Re: [PR] [HUDI-8881] trigger failure earlier in Flink Hudi sink [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12666: URL: https://github.com/apache/hudi/pull/12666#issuecomment-2599033303 ## CI report: * 6194d602858d8d1f6fcdb3c53c5ffca7cd71a881 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

[PR] [HUDI-8881] trigger failure earlier Flink Hudi sink [hudi]

2025-01-17 Thread via GitHub
HuangZhenQiu opened a new pull request, #12666: URL: https://github.com/apache/hudi/pull/12666 ### Change Logs Trigger failure when write failure is detected for flink hudi sink ### Impact The early detection of write failure will prevent checkpoint to complete before hu

[jira] [Updated] (HUDI-8881) Potential dataloss in Flink hudi sink

2025-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-8881: - Labels: pull-request-available (was: ) > Potential dataloss in Flink hudi sink >

Re: [PR] [HUDI-8839] CdcFileGroupIterator use spillable hashmap [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12592: URL: https://github.com/apache/hudi/pull/12592#issuecomment-2599011351 ## CI report: * 423421ec00e72021f081c901ac74891a266b8aa5 UNKNOWN * a5544e3e3d5aa734348b7bfd63820d5b8d98cc33 UNKNOWN * 224d4fe1e0cc28ea3f31a41f2042b28217d8783d Azure: [SUCC

[jira] [Closed] (HUDI-8629) MergeInto w/ Partial updates pulls in fields from source not in assignment clause

2025-01-17 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-8629. - Resolution: Fixed [https://github.com/apache/hudi/commit/d344710878bb11e4a6673179888be0e34

[jira] [Updated] (HUDI-8413) Change the default merge mode to COMMIT_TIME_ORDERING

2025-01-17 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-8413: -- Status: In Progress (was: Patch Available) > Change the default merge mode to COMMIT_TI

Re: [PR] [HUDI-8413] Change the default merge mode to COMMIT_TIME_ORDERING [hudi]

2025-01-17 Thread via GitHub
nsivabalan commented on code in PR #12600: URL: https://github.com/apache/hudi/pull/12600#discussion_r1920596710 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java: ## @@ -179,7 +179,7 @@ public class HoodieWriteConfig extends HoodieCon

Re: [PR] [MINOR] fix up of HUDI-8804 [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12572: URL: https://github.com/apache/hudi/pull/12572#issuecomment-2598972254 ## CI report: * cf22718242150208e843db628ad046a180cca2f3 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2701)

Re: [PR] [MINOR] fix up of HUDI-8804 [hudi]

2025-01-17 Thread via GitHub
hudi-bot commented on PR #12572: URL: https://github.com/apache/hudi/pull/12572#issuecomment-2598969864 ## CI report: * cf22718242150208e843db628ad046a180cca2f3 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=2701)

[jira] [Updated] (HUDI-7483) TestHoodieClientMultiWriter.testMultiWriterWithAsyncTableServicesWithConflict

2025-01-17 Thread Davis Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Zhang updated HUDI-7483: -- Parent: HUDI-8695 Issue Type: Sub-task (was: Bug) > TestHoodieClientMultiWriter.testMultiWriter

[jira] [Updated] (HUDI-7483) TestHoodieClientMultiWriter.testMultiWriterWithAsyncTableServicesWithConflict

2025-01-17 Thread Davis Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Zhang updated HUDI-7483: -- Epic Link: (was: HUDI-4302) > TestHoodieClientMultiWriter.testMultiWriterWithAsyncTableServicesWithC

  1   2   >