[GitHub] [hudi] codope commented on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-07 Thread GitBox
codope commented on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1007218764 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [hudi] hudi-bot commented on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1007219440 ## CI report: * fde98bb87aca9963a377d036f458d83f23a90ad6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-07 Thread GitBox
hudi-bot removed a comment on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1007148515 ## CI report: * fde98bb87aca9963a377d036f458d83f23a90ad6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1007224207 ## CI report: * fde98bb87aca9963a377d036f458d83f23a90ad6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-07 Thread GitBox
hudi-bot removed a comment on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1007219440 ## CI report: * fde98bb87aca9963a377d036f458d83f23a90ad6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1007225773 ## CI report: * fde98bb87aca9963a377d036f458d83f23a90ad6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-07 Thread GitBox
hudi-bot removed a comment on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1007224207 ## CI report: * fde98bb87aca9963a377d036f458d83f23a90ad6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-07 Thread GitBox
hudi-bot removed a comment on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1007225773 ## CI report: * fde98bb87aca9963a377d036f458d83f23a90ad6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1007227397 ## CI report: * fde98bb87aca9963a377d036f458d83f23a90ad6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] danny0405 commented on issue #4508: [SUPPORT]Duplicate Flink Hudi data

2022-01-07 Thread GitBox
danny0405 commented on issue #4508: URL: https://github.com/apache/hudi/issues/4508#issuecomment-1007233408 I said you should use the right code version. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[jira] [Updated] (HUDI-3125) Spark SQL writing timestamp type don't need to disable `spark.sql.datetime.java8API.enabled` manually

2022-01-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3125: - Issue Type: Improvement (was: Bug) > Spark SQL writing timestamp type don't need to disable > `spark.sql

[jira] [Updated] (HUDI-3125) Spark SQL writing timestamp type don't need to disable `spark.sql.datetime.java8API.enabled` manually

2022-01-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3125: - Fix Version/s: (was: 0.11.0) > Spark SQL writing timestamp type don't need to disable > `spark.sql.da

[GitHub] [hudi] manojpec commented on a change in pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

2022-01-07 Thread GitBox
manojpec commented on a change in pull request #4352: URL: https://github.com/apache/hudi/pull/4352#discussion_r780117989 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/bloom/HoodieBloomIndex.java ## @@ -111,13 +124,14 @@ public HoodieBloomInd

[GitHub] [hudi] Limess commented on issue #4525: [SUPPORT] Spark metastore schema evolution broken

2022-01-07 Thread GitBox
Limess commented on issue #4525: URL: https://github.com/apache/hudi/issues/4525#issuecomment-1007247570 I think this is the same issue I raised on JIRA previously https://issues.apache.org/jira/browse/HUDI-2682 -- This is an automated message from the Apache Git Service. To respond to t

[GitHub] [hudi] manojpec commented on a change in pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

2022-01-07 Thread GitBox
manojpec commented on a change in pull request #4352: URL: https://github.com/apache/hudi/pull/4352#discussion_r780119895 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/index/bloom/HoodieBloomMetaIndexBatchCheckFunction.java ## @@ -0,0 +1,155 @@ +/*

[GitHub] [hudi] manojpec commented on a change in pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

2022-01-07 Thread GitBox
manojpec commented on a change in pull request #4352: URL: https://github.com/apache/hudi/pull/4352#discussion_r780120655 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/index/bloom/HoodieBloomMetaIndexBatchCheckFunction.java ## @@ -0,0 +1,155 @@ +/*

[GitHub] [hudi] manojpec commented on a change in pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

2022-01-07 Thread GitBox
manojpec commented on a change in pull request #4352: URL: https://github.com/apache/hudi/pull/4352#discussion_r780125704 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/index/bloom/HoodieBloomMetaIndexBatchCheckFunction.java ## @@ -0,0 +1,155 @@ +/*

[GitHub] [hudi] manojpec commented on a change in pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

2022-01-07 Thread GitBox
manojpec commented on a change in pull request #4352: URL: https://github.com/apache/hudi/pull/4352#discussion_r780126959 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java ## @@ -735,14 +768,19 @@ protected

[GitHub] [hudi] manojpec commented on a change in pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

2022-01-07 Thread GitBox
manojpec commented on a change in pull request #4352: URL: https://github.com/apache/hudi/pull/4352#discussion_r780127079 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/index/bloom/HoodieBloomMetaIndexColStatFunction.java ## @@ -0,0 +1,93 @@ +/* + *

[GitHub] [hudi] dongkelun opened a new pull request #4534: [MINOR] fix typos in DDLExecutor

2022-01-07 Thread GitBox
dongkelun opened a new pull request #4534: URL: https://github.com/apache/hudi/pull/4534 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpo

[GitHub] [hudi] hudi-bot commented on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1007262872 ## CI report: * 9c22d6c689ab1b413f52acbced2ad65062f993ed Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-07 Thread GitBox
hudi-bot removed a comment on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1007227397 ## CI report: * fde98bb87aca9963a377d036f458d83f23a90ad6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4534: [MINOR] fix typos in DDLExecutor

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4534: URL: https://github.com/apache/hudi/pull/4534#issuecomment-1007263210 ## CI report: * 9d58ff8c778fa082bff8808c0343428d5b0801e1 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot removed a comment on pull request #4534: [MINOR] fix typos in DDLExecutor

2022-01-07 Thread GitBox
hudi-bot removed a comment on pull request #4534: URL: https://github.com/apache/hudi/pull/4534#issuecomment-1007263210 ## CI report: * 9d58ff8c778fa082bff8808c0343428d5b0801e1 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4534: [MINOR] fix typos in DDLExecutor

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4534: URL: https://github.com/apache/hudi/pull/4534#issuecomment-1007265007 ## CI report: * 9d58ff8c778fa082bff8808c0343428d5b0801e1 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[jira] [Created] (HUDI-3194) Fix invisible writes(commits) during compaction (HoodieParquetRealtimeInputFormat)

2022-01-07 Thread Yuwei Xiao (Jira)
Yuwei Xiao created HUDI-3194: Summary: Fix invisible writes(commits) during compaction (HoodieParquetRealtimeInputFormat) Key: HUDI-3194 URL: https://issues.apache.org/jira/browse/HUDI-3194 Project: Apach

[jira] [Commented] (HUDI-3157) Remove aws jars from hudi bundles

2022-01-07 Thread Hui An (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17470499#comment-17470499 ] Hui An commented on HUDI-3157: -- If you don't mind, I can help on this ticket :D > Remove aws

[GitHub] [hudi] hudi-bot commented on pull request #4534: [MINOR] fix typos in DDLExecutor

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4534: URL: https://github.com/apache/hudi/pull/4534#issuecomment-1007306426 ## CI report: * 9d58ff8c778fa082bff8808c0343428d5b0801e1 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4534: [MINOR] fix typos in DDLExecutor

2022-01-07 Thread GitBox
hudi-bot removed a comment on pull request #4534: URL: https://github.com/apache/hudi/pull/4534#issuecomment-1007265007 ## CI report: * 9d58ff8c778fa082bff8808c0343428d5b0801e1 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1007329081 ## CI report: * 9c22d6c689ab1b413f52acbced2ad65062f993ed Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-07 Thread GitBox
hudi-bot removed a comment on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1007262872 ## CI report: * 9c22d6c689ab1b413f52acbced2ad65062f993ed Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1007330864 ## CI report: * 9c22d6c689ab1b413f52acbced2ad65062f993ed Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-07 Thread GitBox
hudi-bot removed a comment on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1007329081 ## CI report: * 9c22d6c689ab1b413f52acbced2ad65062f993ed Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1007358374 ## CI report: * 62636d29edcfa61fb66622dd8cb6319ff118771d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-07 Thread GitBox
hudi-bot removed a comment on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1007330864 ## CI report: * 9c22d6c689ab1b413f52acbced2ad65062f993ed Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] XuQianJin-Stars opened a new pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-07 Thread GitBox
XuQianJin-Stars opened a new pull request #4535: URL: https://github.com/apache/hudi/pull/4535 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

[jira] [Updated] (HUDI-3161) Add Call Produce Command for spark sql

2022-01-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3161: - Labels: pull-request-available (was: ) > Add Call Produce Command for spark sql > ---

[GitHub] [hudi] hudi-bot commented on pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1007371885 ## CI report: * 377ada95264d66fc8ed8f81e3bf95ec05a35d450 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot commented on pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1007373692 ## CI report: * 377ada95264d66fc8ed8f81e3bf95ec05a35d450 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-07 Thread GitBox
hudi-bot removed a comment on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1007371885 ## CI report: * 377ada95264d66fc8ed8f81e3bf95ec05a35d450 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] nsivabalan commented on pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-07 Thread GitBox
nsivabalan commented on pull request #4514: URL: https://github.com/apache/hudi/pull/4514#issuecomment-1007379499 yeah, would really appreciate if we can wait until Jan 9 to land this patch. thanks! -- This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [hudi] codope opened a new pull request #4536: [HUDI-3185] HoodieConfig#getBoolean should return false when default …

2022-01-07 Thread GitBox
codope opened a new pull request #4536: URL: https://github.com/apache/hudi/pull/4536 …not set ## What is the purpose of the pull request If user has not set a config and `setDefaults()` is not called before checking the config, then `getBoolean` (or getString, getInt) returns

[jira] [Updated] (HUDI-3185) HoodieConfig getBoolean method returns null instead of default value

2022-01-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3185: - Labels: pull-request-available (was: ) > HoodieConfig getBoolean method returns null instead of d

[GitHub] [hudi] hudi-bot commented on pull request #4536: [HUDI-3185] HoodieConfig#getBoolean should return false when default …

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4536: URL: https://github.com/apache/hudi/pull/4536#issuecomment-1007382826 ## CI report: * d7f3efb72a5260e535336b53d46afe2835fa57ec UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot removed a comment on pull request #4536: [HUDI-3185] HoodieConfig#getBoolean should return false when default …

2022-01-07 Thread GitBox
hudi-bot removed a comment on pull request #4536: URL: https://github.com/apache/hudi/pull/4536#issuecomment-1007382826 ## CI report: * d7f3efb72a5260e535336b53d46afe2835fa57ec UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4536: [HUDI-3185] HoodieConfig#getBoolean should return false when default …

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4536: URL: https://github.com/apache/hudi/pull/4536#issuecomment-1007384674 ## CI report: * d7f3efb72a5260e535336b53d46afe2835fa57ec Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] nsivabalan commented on pull request #4530: [HUDI-3178] Fixing metadata table compaction so as to not include uncommitted data

2022-01-07 Thread GitBox
nsivabalan commented on pull request #4530: URL: https://github.com/apache/hudi/pull/4530#issuecomment-1007385719 @manojpec : Can you review the patch too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] nsivabalan merged pull request #4527: [HUDI-3188] Update quick start guide for Kafka Connect Sink for Hudi

2022-01-07 Thread GitBox
nsivabalan merged pull request #4527: URL: https://github.com/apache/hudi/pull/4527 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[hudi] branch master updated (2467c13 -> 76a7264)

2022-01-07 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 2467c13 [HUDI-3100] Add config for hive conditional sync (#4440) add 76a7264 [HUDI-3188] Update quick start g

[jira] [Resolved] (HUDI-3188) Bridge gaps in Quick Start (demo) guide for Kafka Connect Sink for Hudi

2022-01-07 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan resolved HUDI-3188. --- > Bridge gaps in Quick Start (demo) guide for Kafka Connect Sink for Hudi > --

[GitHub] [hudi] hudi-bot removed a comment on pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-07 Thread GitBox
hudi-bot removed a comment on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1007373692 ## CI report: * 377ada95264d66fc8ed8f81e3bf95ec05a35d450 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] nsivabalan merged pull request #4534: [MINOR] fix typos in DDLExecutor

2022-01-07 Thread GitBox
nsivabalan merged pull request #4534: URL: https://github.com/apache/hudi/pull/4534 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[GitHub] [hudi] hudi-bot commented on pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1007388018 ## CI report: * 377ada95264d66fc8ed8f81e3bf95ec05a35d450 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[hudi] branch master updated (76a7264 -> b1df606)

2022-01-07 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 76a7264 [HUDI-3188] Update quick start guide for Kafka Connect Sink for Hudi (#4527) add b1df606 [MINOR] fix

[GitHub] [hudi] nsivabalan commented on pull request #4533: [HUDI-3192] Spark metastore schema evolution broken

2022-01-07 Thread GitBox
nsivabalan commented on pull request #4533: URL: https://github.com/apache/hudi/pull/4533#issuecomment-1007388310 @xiarixiaoyao : Can you please review this patch. thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [hudi] hudi-bot commented on pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1007389842 ## CI report: * 377ada95264d66fc8ed8f81e3bf95ec05a35d450 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-07 Thread GitBox
hudi-bot removed a comment on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1007388018 ## CI report: * 377ada95264d66fc8ed8f81e3bf95ec05a35d450 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] nsivabalan commented on pull request #4515: [HUDI-3158] Reduce warn logs in Spark SQL INSERT OVERWRITE

2022-01-07 Thread GitBox
nsivabalan commented on pull request #4515: URL: https://github.com/apache/hudi/pull/4515#issuecomment-1007390563 @codope : Can you review this PR. May be there could be some follow ups in other places where we do create similar empty replace commit instants. I -- This is an automated me

[GitHub] [hudi] nsivabalan commented on issue #4525: [SUPPORT] Spark metastore schema evolution broken

2022-01-07 Thread GitBox
nsivabalan commented on issue #4525: URL: https://github.com/apache/hudi/issues/4525#issuecomment-1007391256 Closing this as we have a tracking jira and a patch put up on this end. https://github.com/apache/hudi/pull/4533 thanks for reporting. -- This is an automated message from

[GitHub] [hudi] nsivabalan commented on pull request #4533: [HUDI-3192] Spark metastore schema evolution broken

2022-01-07 Thread GitBox
nsivabalan commented on pull request #4533: URL: https://github.com/apache/hudi/pull/4533#issuecomment-1007392092 @dongkelun : can you please check if HUDI-3192 and https://issues.apache.org/jira/browse/HUDI-2682 are duplicates. if yes, please mark one of them as duplicate and close it.

[GitHub] [hudi] nsivabalan closed issue #4525: [SUPPORT] Spark metastore schema evolution broken

2022-01-07 Thread GitBox
nsivabalan closed issue #4525: URL: https://github.com/apache/hudi/issues/4525 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[GitHub] [hudi] nsivabalan commented on a change in pull request #4485: [HUDI-2947] Fixing checkpoint fetch in detlastreamer

2022-01-07 Thread GitBox
nsivabalan commented on a change in pull request #4485: URL: https://github.com/apache/hudi/pull/4485#discussion_r780253945 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/DeltaSync.java ## @@ -471,6 +468,22 @@ public void refreshTimeline() t

[GitHub] [hudi] hudi-bot commented on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1007400044 ## CI report: * 62636d29edcfa61fb66622dd8cb6319ff118771d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-07 Thread GitBox
hudi-bot removed a comment on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1007358374 ## CI report: * 62636d29edcfa61fb66622dd8cb6319ff118771d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1007402163 ## CI report: * 62636d29edcfa61fb66622dd8cb6319ff118771d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-07 Thread GitBox
hudi-bot removed a comment on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1007400044 ## CI report: * 62636d29edcfa61fb66622dd8cb6319ff118771d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[jira] [Assigned] (HUDI-2682) Spark schema not updated with new columns on hive sync

2022-01-07 Thread Jira
[ https://issues.apache.org/jira/browse/HUDI-2682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 董可伦 reassigned HUDI-2682: - Assignee: 董可伦 > Spark schema not updated with new columns on hive sync >

[jira] [Closed] (HUDI-3192) Spark metastore schema evolution broken

2022-01-07 Thread Jira
[ https://issues.apache.org/jira/browse/HUDI-3192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 董可伦 closed HUDI-3192. - Resolution: Duplicate > Spark metastore schema evolution broken > --- > >

[jira] [Updated] (HUDI-2682) Spark schema not updated with new columns on hive sync

2022-01-07 Thread Jira
[ https://issues.apache.org/jira/browse/HUDI-2682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 董可伦 updated HUDI-2682: -- Component/s: Spark Integration Fix Version/s: 0.11.0 0.10.1 > Spark schema not updated with new

[GitHub] [hudi] hudi-bot commented on pull request #4495: [HUDI-3139] Shade htrace and parquet-avro in presto bundle

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4495: URL: https://github.com/apache/hudi/pull/4495#issuecomment-1007407141 ## CI report: * 3403d8a6a20793c7d2088b11a32fa2abaabcea12 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[jira] [Updated] (HUDI-2682) Spark schema not updated with new columns on hive sync

2022-01-07 Thread Jira
[ https://issues.apache.org/jira/browse/HUDI-2682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 董可伦 updated HUDI-2682: -- Priority: Blocker (was: Minor) > Spark schema not updated with new columns on hive sync > -

[GitHub] [hudi] hudi-bot removed a comment on pull request #4495: [HUDI-3139] Shade htrace and parquet-avro in presto bundle

2022-01-07 Thread GitBox
hudi-bot removed a comment on pull request #4495: URL: https://github.com/apache/hudi/pull/4495#issuecomment-1003993031 ## CI report: * 3403d8a6a20793c7d2088b11a32fa2abaabcea12 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4495: [HUDI-3139] Shade htrace and parquet-avro in presto bundle

2022-01-07 Thread GitBox
hudi-bot removed a comment on pull request #4495: URL: https://github.com/apache/hudi/pull/4495#issuecomment-1007407141 ## CI report: * 3403d8a6a20793c7d2088b11a32fa2abaabcea12 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4495: [HUDI-3139] Shade htrace and parquet-avro in presto bundle

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4495: URL: https://github.com/apache/hudi/pull/4495#issuecomment-1007409425 ## CI report: * 3403d8a6a20793c7d2088b11a32fa2abaabcea12 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] dongkelun commented on pull request #4533: [HUDI-2682] Spark schema not updated with new columns on hive sync

2022-01-07 Thread GitBox
dongkelun commented on pull request #4533: URL: https://github.com/apache/hudi/pull/4533#issuecomment-1007410108 > @dongkelun : can you please check if [HUDI-3192](https://issues.apache.org/jira/browse/HUDI-3192) and https://issues.apache.org/jira/browse/HUDI-2682 are duplicates. if yes, p

[GitHub] [hudi] dongkelun edited a comment on pull request #4533: [HUDI-2682] Spark schema not updated with new columns on hive sync

2022-01-07 Thread GitBox
dongkelun edited a comment on pull request #4533: URL: https://github.com/apache/hudi/pull/4533#issuecomment-1007410108 > @dongkelun : can you please check if [HUDI-3192](https://issues.apache.org/jira/browse/HUDI-3192) and https://issues.apache.org/jira/browse/HUDI-2682 are duplicates. if

[jira] [Updated] (HUDI-2682) Spark schema not updated with new columns on hive sync

2022-01-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2682: - Labels: pull-request-available (was: ) > Spark schema not updated with new columns on hive sync >

[GitHub] [hudi] hudi-bot removed a comment on pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-07 Thread GitBox
hudi-bot removed a comment on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1007389842 ## CI report: * 377ada95264d66fc8ed8f81e3bf95ec05a35d450 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1007413786 ## CI report: * 377ada95264d66fc8ed8f81e3bf95ec05a35d450 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #4533: [HUDI-2682] Spark schema not updated with new columns on hive sync

2022-01-07 Thread GitBox
xiarixiaoyao commented on a change in pull request #4533: URL: https://github.com/apache/hudi/pull/4533#discussion_r780270395 ## File path: hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java ## @@ -251,10 +251,8 @@ private boolean syncSchema(String t

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #4533: [HUDI-2682] Spark schema not updated with new columns on hive sync

2022-01-07 Thread GitBox
xiarixiaoyao commented on a change in pull request #4533: URL: https://github.com/apache/hudi/pull/4533#discussion_r780276528 ## File path: hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java ## @@ -251,10 +251,8 @@ private boolean syncSchema(String t

[GitHub] [hudi] dongkelun commented on a change in pull request #4533: [HUDI-2682] Spark schema not updated with new columns on hive sync

2022-01-07 Thread GitBox
dongkelun commented on a change in pull request #4533: URL: https://github.com/apache/hudi/pull/4533#discussion_r780276868 ## File path: hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java ## @@ -251,10 +251,8 @@ private boolean syncSchema(String tabl

[GitHub] [hudi] dongkelun commented on a change in pull request #4533: [HUDI-2682] Spark schema not updated with new columns on hive sync

2022-01-07 Thread GitBox
dongkelun commented on a change in pull request #4533: URL: https://github.com/apache/hudi/pull/4533#discussion_r780280392 ## File path: hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java ## @@ -251,10 +251,8 @@ private boolean syncSchema(String tabl

[GitHub] [hudi] nsivabalan commented on pull request #4485: [HUDI-2947] Fixing checkpoint fetch in detlastreamer

2022-01-07 Thread GitBox
nsivabalan commented on pull request #4485: URL: https://github.com/apache/hudi/pull/4485#issuecomment-1007430788 @codope : addressed all comments -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #4533: [HUDI-2682] Spark schema not updated with new columns on hive sync

2022-01-07 Thread GitBox
xiarixiaoyao commented on a change in pull request #4533: URL: https://github.com/apache/hudi/pull/4533#discussion_r780282071 ## File path: hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java ## @@ -251,10 +251,8 @@ private boolean syncSchema(String t

[GitHub] [hudi] hudi-bot removed a comment on pull request #4485: [HUDI-2947] Fixing checkpoint fetch in detlastreamer

2022-01-07 Thread GitBox
hudi-bot removed a comment on pull request #4485: URL: https://github.com/apache/hudi/pull/4485#issuecomment-1003399247 ## CI report: * 4987bc78d52f97537a77e4dbdc7e0b1e294ee3ec Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4485: [HUDI-2947] Fixing checkpoint fetch in detlastreamer

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4485: URL: https://github.com/apache/hudi/pull/4485#issuecomment-1007433563 ## CI report: * 4987bc78d52f97537a77e4dbdc7e0b1e294ee3ec Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] jardel-lima commented on issue #3879: [SUPPORT] Incomplete Table Migration

2022-01-07 Thread GitBox
jardel-lima commented on issue #3879: URL: https://github.com/apache/hudi/issues/3879#issuecomment-1007434803 Hi @nsivabalan. [HERE](https://drive.google.com/file/d/1RsesivvlLUZ9dZh7WbaGJJpnqIcWNbso/view?usp=sharing) is the dataset used to replicate this problem. The file is not public

[GitHub] [hudi] jardel-lima edited a comment on issue #3879: [SUPPORT] Incomplete Table Migration

2022-01-07 Thread GitBox
jardel-lima edited a comment on issue #3879: URL: https://github.com/apache/hudi/issues/3879#issuecomment-1007434803 Hi @nsivabalan. [HERE](https://drive.google.com/file/d/1RsesivvlLUZ9dZh7WbaGJJpnqIcWNbso/view?usp=sharing) is the dataset used to replicate this problem. The file is not

[GitHub] [hudi] hudi-bot removed a comment on pull request #4485: [HUDI-2947] Fixing checkpoint fetch in detlastreamer

2022-01-07 Thread GitBox
hudi-bot removed a comment on pull request #4485: URL: https://github.com/apache/hudi/pull/4485#issuecomment-1007433563 ## CI report: * 4987bc78d52f97537a77e4dbdc7e0b1e294ee3ec Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] dongkelun commented on a change in pull request #4533: [HUDI-2682] Spark schema not updated with new columns on hive sync

2022-01-07 Thread GitBox
dongkelun commented on a change in pull request #4533: URL: https://github.com/apache/hudi/pull/4533#discussion_r780286195 ## File path: hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java ## @@ -251,10 +251,8 @@ private boolean syncSchema(String tabl

[GitHub] [hudi] jardel-lima edited a comment on issue #3879: [SUPPORT] Incomplete Table Migration

2022-01-07 Thread GitBox
jardel-lima edited a comment on issue #3879: URL: https://github.com/apache/hudi/issues/3879#issuecomment-1007434803 Hi @nsivabalan. [HERE](https://drive.google.com/file/d/1RsesivvlLUZ9dZh7WbaGJJpnqIcWNbso/view?usp=sharing) is the dataset used to replicate this problem. The file is not

[GitHub] [hudi] hudi-bot commented on pull request #4485: [HUDI-2947] Fixing checkpoint fetch in detlastreamer

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4485: URL: https://github.com/apache/hudi/pull/4485#issuecomment-1007435713 ## CI report: * 4987bc78d52f97537a77e4dbdc7e0b1e294ee3ec Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] parisni commented on issue #4525: [SUPPORT] Spark metastore schema evolution broken

2022-01-07 Thread GitBox
parisni commented on issue #4525: URL: https://github.com/apache/hudi/issues/4525#issuecomment-1007436024 @Limess @nsivabalan the issue is independant from hudi I guess. CF my [spark-user mail](https://lists.apache.org/thread/9mmrnc5o7w42z723s2yqgcrdpwwtts3x) -- This is an automated mes

[GitHub] [hudi] nsivabalan commented on pull request #4489: [HUDI-3135] Fix Delete partitions with metadata table and fix show partitions in spark sql

2022-01-07 Thread GitBox
nsivabalan commented on pull request #4489: URL: https://github.com/apache/hudi/pull/4489#issuecomment-1007436610 @XuQianJin-Stars : may I know when you plan to address the feedback. We are looking to get this in for 0.10.1. would appreciate if you can find time to address them sooner. Let

[GitHub] [hudi] nsivabalan commented on pull request #4519: [HUDI-3180] Include files from completed commits while bootstrapping metadata table

2022-01-07 Thread GitBox
nsivabalan commented on pull request #4519: URL: https://github.com/apache/hudi/pull/4519#issuecomment-1007437335 @manojpec : can you review the patch please. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [hudi] XuQianJin-Stars commented on pull request #4489: [HUDI-3135] Fix Delete partitions with metadata table and fix show partitions in spark sql

2022-01-07 Thread GitBox
XuQianJin-Stars commented on pull request #4489: URL: https://github.com/apache/hudi/pull/4489#issuecomment-1007438849 > @XuQianJin-Stars : may I know when you plan to address the feedback. We are looking to get this in for 0.10.1. would appreciate if you can find time to address them soon

[GitHub] [hudi] hudi-bot commented on pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-07 Thread GitBox
hudi-bot commented on pull request #4514: URL: https://github.com/apache/hudi/pull/4514#issuecomment-1007440136 ## CI report: * ddc3af0c32bafef6b10c32c43132df32a5f7d83c UNKNOWN * e1ba726105dfa7ae07d802546c71a0cf1ad8b172 UNKNOWN * 306e7d462959e0249e230f60c2e9ea6602342e08 UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-07 Thread GitBox
hudi-bot removed a comment on pull request #4514: URL: https://github.com/apache/hudi/pull/4514#issuecomment-1006428061 ## CI report: * ddc3af0c32bafef6b10c32c43132df32a5f7d83c UNKNOWN * e1ba726105dfa7ae07d802546c71a0cf1ad8b172 UNKNOWN * 306e7d462959e0249e230f60c2e9ea6602342

[GitHub] [hudi] leesf commented on pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-07 Thread GitBox
leesf commented on pull request #4514: URL: https://github.com/apache/hudi/pull/4514#issuecomment-1007441054 > @leesf few questions. > > 1. What do the hudi-spark2-extensions/hudi-spark3-extensions do? What code would these have in the future? > 2. Users may have spark jobs that d

  1   2   3   4   5   >