[GitHub] [hudi] dongkelun commented on pull request #4083: [HUDI-2837] The original hoodie.table.name should be maintained in Spark SQL

2022-01-09 Thread GitBox
dongkelun commented on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1008248624 > @dongkelun LGTM, just left some minor comments. @xushiyan Further review whether this strategy makes sense. Except for two doubts, the others have been submitted for mod

[GitHub] [hudi] YannByron commented on pull request #3420: [HUDI-2283] Support Clustering Command For Spark Sql

2022-01-09 Thread GitBox
YannByron commented on pull request #3420: URL: https://github.com/apache/hudi/pull/3420#issuecomment-1008248938 @nsivabalan @yihua If no one take this up, i am glad to. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] hudi-bot removed a comment on pull request #4083: [HUDI-2837] The original hoodie.table.name should be maintained in Spark SQL

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1008247208 ## CI report: * 00221c82e8b1693280fd72625eafcd503d54323c UNKNOWN * 46053bb143d1fd1274ac466197cc9361708e738b UNKNOWN * 020118c4b169d47e668f37240388e8d1bbdfa

[GitHub] [hudi] hudi-bot commented on pull request #4083: [HUDI-2837] The original hoodie.table.name should be maintained in Spark SQL

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1008253012 ## CI report: * 00221c82e8b1693280fd72625eafcd503d54323c UNKNOWN * 46053bb143d1fd1274ac466197cc9361708e738b UNKNOWN * 020118c4b169d47e668f37240388e8d1bbdfad70 Azur

[GitHub] [hudi] yihua commented on a change in pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

2022-01-09 Thread GitBox
yihua commented on a change in pull request #4078: URL: https://github.com/apache/hudi/pull/4078#discussion_r780743992 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieCompactionConfig.java ## @@ -249,6 +249,24 @@ + "record siz

[GitHub] [hudi] yihua commented on a change in pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

2022-01-09 Thread GitBox
yihua commented on a change in pull request #4078: URL: https://github.com/apache/hudi/pull/4078#discussion_r780750369 ## File path: hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/io/TestHoodieTimelineArchiveLog.java ## @@ -174,6 +205,82 @@ public void testArchive

[GitHub] [hudi] YannByron commented on issue #4477: [SUPPORT]using spark on TimestampBasedKeyGenerator has no result when query by partition column

2022-01-09 Thread GitBox
YannByron commented on issue #4477: URL: https://github.com/apache/hudi/issues/4477#issuecomment-1008254173 @nsivabalan I haven't created a ticket yet. You can create it and assign to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [hudi] yihua commented on pull request #4441: [HUDI-3085] improve bulk insert partitioner abstraction

2022-01-09 Thread GitBox
yihua commented on pull request #4441: URL: https://github.com/apache/hudi/pull/4441#issuecomment-1008254240 @YuweiXiao Could you rebase your PR on master to resolve the conflicts? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[jira] [Assigned] (HUDI-310) DynamoDB/Kinesis Change Capture using Delta Streamer

2022-01-09 Thread Vinay (Jira)
[ https://issues.apache.org/jira/browse/HUDI-310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinay reassigned HUDI-310: -- Assignee: Vinay (was: Suneel Marthi) > DynamoDB/Kinesis Change Capture using Delta Streamer > -

[jira] [Commented] (HUDI-310) DynamoDB/Kinesis Change Capture using Delta Streamer

2022-01-09 Thread Vinay (Jira)
[ https://issues.apache.org/jira/browse/HUDI-310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17471309#comment-17471309 ] Vinay commented on HUDI-310: [~vinoth]  I remember discussing about this, sry it went to backlo

[GitHub] [hudi] yihua commented on a change in pull request #4346: [HUDI-3045] New clustering regex match config to choose partitions when building clustering plan

2022-01-09 Thread GitBox
yihua commented on a change in pull request #4346: URL: https://github.com/apache/hudi/pull/4346#discussion_r780751859 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieClusteringConfig.java ## @@ -70,6 +70,12 @@ .sinceVersion("0.7.

[jira] [Commented] (HUDI-3065) spark auto partition discovery does not work from 0.9.0

2022-01-09 Thread Forward Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17471311#comment-17471311 ] Forward Xu commented on HUDI-3065: -- hi [~shivnarayan] I have found this problem, just nee

[GitHub] [hudi] XuQianJin-Stars commented on issue #3984: [SUPPORT] Upgrade from 0.8.0 to 0.9.0 removes functionality and decreases performance

2022-01-09 Thread GitBox
XuQianJin-Stars commented on issue #3984: URL: https://github.com/apache/hudi/issues/3984#issuecomment-1008258618 hi @cb149 @nsivabalan @xushiyan I have found this problem, just need to `set hoodie.file.index.enable=false` to work ``` val tripsSnapshotDF = spark.read.format("hudi

[GitHub] [hudi] yihua commented on issue #4230: [SUPPORT] org.apache.hudi.exception.HoodieRemoteException: Failed to create marker file

2022-01-09 Thread GitBox
yihua commented on issue #4230: URL: https://github.com/apache/hudi/issues/4230#issuecomment-1008258786 @nsivabalan There might be an issue related to how the timeline server handles the timeout, but this requires a larger fix involving designing a prompt timeout scheme besides Javalin tim

[GitHub] [hudi] yihua merged pull request #4532: [Minor]Fix some code style based on check-sytle plugin

2022-01-09 Thread GitBox
yihua merged pull request #4532: URL: https://github.com/apache/hudi/pull/4532 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[hudi] branch master updated (3679070 -> cf362fb)

2022-01-09 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 3679070 [HUDI-3125] spark-sql write timestamp directly (#4471) add cf362fb [MINOR] Fix some code style issues bas

[GitHub] [hudi] hudi-bot removed a comment on pull request #4083: [HUDI-2837] The original hoodie.table.name should be maintained in Spark SQL

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1008253012 ## CI report: * 00221c82e8b1693280fd72625eafcd503d54323c UNKNOWN * 46053bb143d1fd1274ac466197cc9361708e738b UNKNOWN * 020118c4b169d47e668f37240388e8d1bbdfa

[GitHub] [hudi] hudi-bot commented on pull request #4083: [HUDI-2837] The original hoodie.table.name should be maintained in Spark SQL

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1008260737 ## CI report: * 00221c82e8b1693280fd72625eafcd503d54323c UNKNOWN * 46053bb143d1fd1274ac466197cc9361708e738b UNKNOWN * 2722bfcfd29a95f27338c1c8b026185472eefba0 UNKN

[jira] [Created] (HUDI-3199) Hive sync config unification

2022-01-09 Thread cdmikechen (Jira)
cdmikechen created HUDI-3199: Summary: Hive sync config unification Key: HUDI-3199 URL: https://issues.apache.org/jira/browse/HUDI-3199 Project: Apache Hudi Issue Type: Bug Components:

[GitHub] [hudi] cdmikechen commented on pull request #4458: [HUDI-3112] Fix KafkaConnect can not sync to Hive Problem

2022-01-09 Thread GitBox
cdmikechen commented on pull request #4458: URL: https://github.com/apache/hudi/pull/4458#issuecomment-1008262491 @yihua Hi~ I've moved `buildSyncConfig` method to `KafkaConnectUtils` and open a new issue -- This is an automated messa

[GitHub] [hudi] hudi-bot removed a comment on pull request #4458: [HUDI-3112] Fix KafkaConnect can not sync to Hive Problem

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4458: URL: https://github.com/apache/hudi/pull/4458#issuecomment-1001957335 ## CI report: * c60da50ce53a0b9ff05677f8e7243f42d39fdbd4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4458: [HUDI-3112] Fix KafkaConnect can not sync to Hive Problem

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4458: URL: https://github.com/apache/hudi/pull/4458#issuecomment-1008262905 ## CI report: * c60da50ce53a0b9ff05677f8e7243f42d39fdbd4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4458: [HUDI-3112] Fix KafkaConnect can not sync to Hive Problem

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4458: URL: https://github.com/apache/hudi/pull/4458#issuecomment-1008262905 ## CI report: * c60da50ce53a0b9ff05677f8e7243f42d39fdbd4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4458: [HUDI-3112] Fix KafkaConnect can not sync to Hive Problem

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4458: URL: https://github.com/apache/hudi/pull/4458#issuecomment-1008263265 ## CI report: * c60da50ce53a0b9ff05677f8e7243f42d39fdbd4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] xushiyan merged pull request #4542: [HUDI-3157] Remove aws jars from hudi bundles

2022-01-09 Thread GitBox
xushiyan merged pull request #4542: URL: https://github.com/apache/hudi/pull/4542 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr

[hudi] branch master updated: [HUDI-3157] Remove aws jars from hudi bundles (#4542)

2022-01-09 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 977d3c6 [HUDI-3157] Remove aws jars from hudi b

[jira] [Closed] (HUDI-3157) Remove aws jars from hudi bundles

2022-01-09 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-3157. Resolution: Fixed > Remove aws jars from hudi bundles > - > >

[GitHub] [hudi] hudi-bot removed a comment on pull request #4458: [HUDI-3112] Fix KafkaConnect can not sync to Hive Problem

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4458: URL: https://github.com/apache/hudi/pull/4458#issuecomment-1008263265 ## CI report: * c60da50ce53a0b9ff05677f8e7243f42d39fdbd4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4458: [HUDI-3112] Fix KafkaConnect can not sync to Hive Problem

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4458: URL: https://github.com/apache/hudi/pull/4458#issuecomment-1008271206 ## CI report: * 8d1205d795799594aefe852a6ddeeb13fb38072f Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] aznwarmonkey commented on issue #4541: [SUPPORT] NullPointerException while writing Bulk ingest table

2022-01-09 Thread GitBox
aznwarmonkey commented on issue #4541: URL: https://github.com/apache/hudi/issues/4541#issuecomment-1008283680 > Hi, After making the suggested changes as mentioned above, I am getting this exception thrown immediately: ```bash py4j.protocol.Py4JJavaError: An error

[GitHub] [hudi] aznwarmonkey edited a comment on issue #4541: [SUPPORT] NullPointerException while writing Bulk ingest table

2022-01-09 Thread GitBox
aznwarmonkey edited a comment on issue #4541: URL: https://github.com/apache/hudi/issues/4541#issuecomment-1008283680 > Hi, After making the suggested changes as mentioned above, I am getting this exception thrown immediately: ```bash py4j.protocol.Py4JJavaError: A

[GitHub] [hudi] nsivabalan commented on issue #4238: [SUPPORT]Merge cow partitioned files manually

2022-01-09 Thread GitBox
nsivabalan commented on issue #4238: URL: https://github.com/apache/hudi/issues/4238#issuecomment-1008300322 Closing the issue. Feel free to re-open or raise a new one if you need any more assistant. thanks! -- This is an automated message from the Apache Git Service. To respond to the m

[GitHub] [hudi] nsivabalan closed issue #4238: [SUPPORT]Merge cow partitioned files manually

2022-01-09 Thread GitBox
nsivabalan closed issue #4238: URL: https://github.com/apache/hudi/issues/4238 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[GitHub] [hudi] hudi-bot commented on pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1008306961 ## CI report: * 28255878b23516bf4f6829a00ec8b7f7c5e1bb92 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1007886423 ## CI report: * 28255878b23516bf4f6829a00ec8b7f7c5e1bb92 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1008307474 ## CI report: * 28255878b23516bf4f6829a00ec8b7f7c5e1bb92 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1008306961 ## CI report: * 28255878b23516bf4f6829a00ec8b7f7c5e1bb92 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] aznwarmonkey edited a comment on issue #4541: [SUPPORT] NullPointerException while writing Bulk ingest table

2022-01-09 Thread GitBox
aznwarmonkey edited a comment on issue #4541: URL: https://github.com/apache/hudi/issues/4541#issuecomment-1008283680 > Hi, After making the suggested changes as mentioned above, I am getting this exception thrown immediately: ```bash py4j.protocol.Py4JJavaError: A

[GitHub] [hudi] xuzifu666 opened a new pull request #4543: [MINOR] refactor HoodieAvroWriteSupport with remove unused static var

2022-01-09 Thread GitBox
xuzifu666 opened a new pull request #4543: URL: https://github.com/apache/hudi/pull/4543 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpo

[GitHub] [hudi] xuzifu666 commented on pull request #4543: [MINOR] refactor HoodieAvroWriteSupport with remove unused static var

2022-01-09 Thread GitBox
xuzifu666 commented on pull request #4543: URL: https://github.com/apache/hudi/pull/4543#issuecomment-1008310992 @yihua have a review please, thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hudi] hudi-bot commented on pull request #4543: [MINOR] refactor HoodieAvroWriteSupport with remove unused static var

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4543: URL: https://github.com/apache/hudi/pull/4543#issuecomment-1008311425 ## CI report: * 29d96ce74dcf16be8d6652d3e02c508bdca7102a UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot commented on pull request #4543: [MINOR] refactor HoodieAvroWriteSupport with remove unused static var

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4543: URL: https://github.com/apache/hudi/pull/4543#issuecomment-1008311963 ## CI report: * 29d96ce74dcf16be8d6652d3e02c508bdca7102a Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4543: [MINOR] refactor HoodieAvroWriteSupport with remove unused static var

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4543: URL: https://github.com/apache/hudi/pull/4543#issuecomment-1008311425 ## CI report: * 29d96ce74dcf16be8d6652d3e02c508bdca7102a UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot removed a comment on pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1008307474 ## CI report: * 28255878b23516bf4f6829a00ec8b7f7c5e1bb92 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1008313540 ## CI report: * 28255878b23516bf4f6829a00ec8b7f7c5e1bb92 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1008314120 ## CI report: * 27feb70ead8e4c7ab24c978f570f098818e662f3 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1008313540 ## CI report: * 28255878b23516bf4f6829a00ec8b7f7c5e1bb92 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1008314681 ## CI report: * 27feb70ead8e4c7ab24c978f570f098818e662f3 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1008314120 ## CI report: * 27feb70ead8e4c7ab24c978f570f098818e662f3 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[jira] [Updated] (HUDI-3161) Add Call Produce Command for spark sql

2022-01-09 Thread Forward Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Forward Xu updated HUDI-3161: - Description: example {code:java} // code placeholder # Produce1 call show_commits_metadata(table => 'test

[jira] [Updated] (HUDI-3161) Add Call Produce Command for spark sql

2022-01-09 Thread Forward Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Forward Xu updated HUDI-3161: - Description: example {code:java} // code placeholder # Produce1 call call show_commits(table => 'test_hudi

[GitHub] [hudi] XuQianJin-Stars commented on pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-09 Thread GitBox
XuQianJin-Stars commented on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1008315343 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [hudi] hudi-bot removed a comment on pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1008314681 ## CI report: * 27feb70ead8e4c7ab24c978f570f098818e662f3 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4535: [HUDI-3161] Add Call Produce Command for spark sql

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1008315759 ## CI report: * 27feb70ead8e4c7ab24c978f570f098818e662f3 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot commented on pull request #4535: [WIP][HUDI-3161] Add Call Produce Command for spark sql

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1008318398 ## CI report: * 27feb70ead8e4c7ab24c978f570f098818e662f3 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #4535: [WIP][HUDI-3161] Add Call Produce Command for spark sql

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1008315759 ## CI report: * 27feb70ead8e4c7ab24c978f570f098818e662f3 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot removed a comment on pull request #4543: [MINOR] refactor HoodieAvroWriteSupport with remove unused static var

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4543: URL: https://github.com/apache/hudi/pull/4543#issuecomment-1008311963 ## CI report: * 29d96ce74dcf16be8d6652d3e02c508bdca7102a Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4543: [MINOR] refactor HoodieAvroWriteSupport with remove unused static var

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4543: URL: https://github.com/apache/hudi/pull/4543#issuecomment-1008321817 ## CI report: * 29d96ce74dcf16be8d6652d3e02c508bdca7102a Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4517: [HUDI-3009] making some fixes to S3 incremental source

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4517: URL: https://github.com/apache/hudi/pull/4517#issuecomment-1006196711 ## CI report: * b01dae5b9f04437dd388f36442e3acdd0bed7daa Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4517: [HUDI-3009] making some fixes to S3 incremental source

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4517: URL: https://github.com/apache/hudi/pull/4517#issuecomment-1008326760 ## CI report: * b01dae5b9f04437dd388f36442e3acdd0bed7daa Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #4517: [HUDI-3009] making some fixes to S3 incremental source

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4517: URL: https://github.com/apache/hudi/pull/4517#issuecomment-1008327323 ## CI report: * b01dae5b9f04437dd388f36442e3acdd0bed7daa Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4517: [HUDI-3009] making some fixes to S3 incremental source

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4517: URL: https://github.com/apache/hudi/pull/4517#issuecomment-1008326760 ## CI report: * b01dae5b9f04437dd388f36442e3acdd0bed7daa Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4535: [WIP][HUDI-3161] Add Call Produce Command for spark sql

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1008318398 ## CI report: * 27feb70ead8e4c7ab24c978f570f098818e662f3 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4535: [WIP][HUDI-3161] Add Call Produce Command for spark sql

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1008328684 ## CI report: * 49b18f6d40a8b859927dcc9d606d40fd4162f0b1 UNKNOWN * 450ccaa4c73197ad56f26c37260f66fc27873f36 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot commented on pull request #4535: [WIP][HUDI-3161] Add Call Produce Command for spark sql

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1008329300 ## CI report: * 49b18f6d40a8b859927dcc9d606d40fd4162f0b1 UNKNOWN * 450ccaa4c73197ad56f26c37260f66fc27873f36 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4535: [WIP][HUDI-3161] Add Call Produce Command for spark sql

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4535: URL: https://github.com/apache/hudi/pull/4535#issuecomment-1008328684 ## CI report: * 49b18f6d40a8b859927dcc9d606d40fd4162f0b1 UNKNOWN * 450ccaa4c73197ad56f26c37260f66fc27873f36 Azure: [FAILURE](https://dev.azure.com/apache-hud

[jira] [Commented] (HUDI-1628) [Umbrella] Improve data locality during ingestion

2022-01-09 Thread Thirumalai Raj R (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17471378#comment-17471378 ] Thirumalai Raj R commented on HUDI-1628: [~vinoth] , sorry I am bit busy with my s

[GitHub] [hudi] hudi-bot removed a comment on pull request #4517: [HUDI-3009] making some fixes to S3 incremental source

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4517: URL: https://github.com/apache/hudi/pull/4517#issuecomment-1008327323 ## CI report: * b01dae5b9f04437dd388f36442e3acdd0bed7daa Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4517: [HUDI-3009] making some fixes to S3 incremental source

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4517: URL: https://github.com/apache/hudi/pull/4517#issuecomment-1008338496 ## CI report: * fa0f0ce771ef6741496a2c87620d4cc257053f48 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] nsivabalan merged pull request #4517: [HUDI-3009] making some fixes to S3 incremental source

2022-01-09 Thread GitBox
nsivabalan merged pull request #4517: URL: https://github.com/apache/hudi/pull/4517 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[hudi] branch master updated (977d3c6 -> 604d988)

2022-01-09 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 977d3c6 [HUDI-3157] Remove aws jars from hudi bundles (#4542) add 604d988 [HUDI-3009] making some fixes to S3

[GitHub] [hudi] nsivabalan commented on pull request #4540: [HUDI-3194][WIP] fix MOR snapshot query (HIVE) during compaction

2022-01-09 Thread GitBox
nsivabalan commented on pull request #4540: URL: https://github.com/apache/hudi/pull/4540#issuecomment-1008343888 @YuweiXiao : I see "WIP" in title. Is the patch good to review or is it still being worked upon ? -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] hudi-bot commented on pull request #4406: [HUDI-3030] InProcessLockPovider as default when any async servcies enabled with no lock provider override

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4406: URL: https://github.com/apache/hudi/pull/4406#issuecomment-1008356396 ## CI report: * 8d492ba1b84ee7f27d2ffd5d76b1e1e5c0b8de75 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4406: [HUDI-3030] InProcessLockPovider as default when any async servcies enabled with no lock provider override

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4406: URL: https://github.com/apache/hudi/pull/4406#issuecomment-1006024460 ## CI report: * 8d492ba1b84ee7f27d2ffd5d76b1e1e5c0b8de75 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4406: [HUDI-3030] InProcessLockPovider as default when any async servcies enabled with no lock provider override

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4406: URL: https://github.com/apache/hudi/pull/4406#issuecomment-1008356903 ## CI report: * 8d492ba1b84ee7f27d2ffd5d76b1e1e5c0b8de75 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4406: [HUDI-3030] InProcessLockPovider as default when any async servcies enabled with no lock provider override

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4406: URL: https://github.com/apache/hudi/pull/4406#issuecomment-1008356396 ## CI report: * 8d492ba1b84ee7f27d2ffd5d76b1e1e5c0b8de75 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] nsivabalan commented on pull request #4015: [HUDI-2780] Fix the issue of Mor log skipping complete blocks when reading data

2022-01-09 Thread GitBox
nsivabalan commented on pull request #4015: URL: https://github.com/apache/hudi/pull/4015#issuecomment-1008360786 @alexeykudinkin : Is there any follow up on this patch. Let's sync up sometime to see if we know what's the right fix and if we can squeeze it in for 0.10.1. -- This is

[jira] [Commented] (HUDI-2426) spark sql extensions breaks read.table from metastore

2022-01-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17471459#comment-17471459 ] sivabalan narayanan commented on HUDI-2426: --- Hey [~parisni] : Can you please upd

[GitHub] [hudi] hudi-bot removed a comment on pull request #4406: [HUDI-3030] InProcessLockPovider as default when any async servcies enabled with no lock provider override

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4406: URL: https://github.com/apache/hudi/pull/4406#issuecomment-1008356903 ## CI report: * 8d492ba1b84ee7f27d2ffd5d76b1e1e5c0b8de75 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4406: [HUDI-3030] InProcessLockPovider as default when any async servcies enabled with no lock provider override

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4406: URL: https://github.com/apache/hudi/pull/4406#issuecomment-1008366567 ## CI report: * c02a5a55b6ab014e1aa9891557cf8febd416d3a3 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] yihua merged pull request #4458: [HUDI-3112] Fix KafkaConnect can not sync to Hive Problem

2022-01-09 Thread GitBox
yihua merged pull request #4458: URL: https://github.com/apache/hudi/pull/4458 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[hudi] branch master updated (604d988 -> e9a7f49)

2022-01-09 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 604d988 [HUDI-3009] making some fixes to S3 incremental source (#4517) add e9a7f49 [HUDI-3112] Fix KafkaConnect c

[GitHub] [hudi] manojpec commented on a change in pull request #4519: [HUDI-3180] Include files from completed commits while bootstrapping metadata table

2022-01-09 Thread GitBox
manojpec commented on a change in pull request #4519: URL: https://github.com/apache/hudi/pull/4519#discussion_r780847273 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java ## @@ -746,9 +746,16 @@ protected

[GitHub] [hudi] manojpec commented on a change in pull request #4530: [HUDI-3178] Fixing metadata table compaction so as to not include uncommitted data

2022-01-09 Thread GitBox
manojpec commented on a change in pull request #4530: URL: https://github.com/apache/hudi/pull/4530#discussion_r780847869 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java ## @@ -689,7 +689,7 @@ protected v

[jira] [Updated] (HUDI-3158) Reduce warn logs in Spark SQL INSERT OVERWRITE

2022-01-09 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3158: - Fix Version/s: 0.11.0 (was: 0.10.1) > Reduce warn logs in Spark SQL INSERT OVERWRIT

[jira] [Updated] (HUDI-3135) Fix Show Partitions Command's Result after drop partition

2022-01-09 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3135: - Story Points: 1 (was: 0.5) > Fix Show Partitions Command's Result after drop partition >

[jira] [Commented] (HUDI-2426) spark sql extensions breaks read.table from metastore

2022-01-09 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17471622#comment-17471622 ] Raymond Xu commented on HUDI-2426: -- I verified with Hudi master (0.11.0-SNAPSHOT) against

[jira] [Updated] (HUDI-3065) spark auto partition discovery does not work from 0.9.0

2022-01-09 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3065: - Description: with 0.8.0, if partition is of the format  "/partitionKey=partitionValue", Spark auto partit

[GitHub] [hudi] YuweiXiao commented on pull request #4540: [HUDI-3194][WIP] fix MOR snapshot query (HIVE) during compaction

2022-01-09 Thread GitBox
YuweiXiao commented on pull request #4540: URL: https://github.com/apache/hudi/pull/4540#issuecomment-1008479468 > @YuweiXiao : I see "WIP" in title. Is the patch good to review or is it still being worked upon ? Yes, it is still in process, as there are failed UT need to be fixed.

[GitHub] [hudi] YuweiXiao commented on pull request #4441: [HUDI-3085] improve bulk insert partitioner abstraction

2022-01-09 Thread GitBox
YuweiXiao commented on pull request #4441: URL: https://github.com/apache/hudi/pull/4441#issuecomment-1008480968 > @YuweiXiao Could you rebase your PR on master to resolve the conflicts? Thanks for the reminder. The conflict has been resolved. -- This is an automated message from t

[GitHub] [hudi] hudi-bot removed a comment on pull request #4441: [HUDI-3085] improve bulk insert partitioner abstraction

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4441: URL: https://github.com/apache/hudi/pull/4441#issuecomment-1000683899 ## CI report: * 10b433d337b81dd2c40deb324eaacd62fe1e5a75 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4441: [HUDI-3085] improve bulk insert partitioner abstraction

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4441: URL: https://github.com/apache/hudi/pull/4441#issuecomment-1008481326 ## CI report: * 10b433d337b81dd2c40deb324eaacd62fe1e5a75 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4441: [HUDI-3085] improve bulk insert partitioner abstraction

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4441: URL: https://github.com/apache/hudi/pull/4441#issuecomment-1008481326 ## CI report: * 10b433d337b81dd2c40deb324eaacd62fe1e5a75 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4441: [HUDI-3085] improve bulk insert partitioner abstraction

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4441: URL: https://github.com/apache/hudi/pull/4441#issuecomment-1008482149 ## CI report: * 10b433d337b81dd2c40deb324eaacd62fe1e5a75 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4441: [HUDI-3085] improve bulk insert partitioner abstraction

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4441: URL: https://github.com/apache/hudi/pull/4441#issuecomment-1008482149 ## CI report: * 10b433d337b81dd2c40deb324eaacd62fe1e5a75 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4441: [HUDI-3085] improve bulk insert partitioner abstraction

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4441: URL: https://github.com/apache/hudi/pull/4441#issuecomment-1008482862 ## CI report: * 1277b45508e2b713a3c8416a87893b1d059c375a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4441: [HUDI-3085] improve bulk insert partitioner abstraction

2022-01-09 Thread GitBox
hudi-bot removed a comment on pull request #4441: URL: https://github.com/apache/hudi/pull/4441#issuecomment-1008482862 ## CI report: * 1277b45508e2b713a3c8416a87893b1d059c375a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4441: [HUDI-3085] improve bulk insert partitioner abstraction

2022-01-09 Thread GitBox
hudi-bot commented on pull request #4441: URL: https://github.com/apache/hudi/pull/4441#issuecomment-1008483743 ## CI report: * 1277b45508e2b713a3c8416a87893b1d059c375a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[jira] [Created] (HUDI-3200) File Index config affects partition fields shown in printSchema results

2022-01-09 Thread Raymond Xu (Jira)
Raymond Xu created HUDI-3200: Summary: File Index config affects partition fields shown in printSchema results Key: HUDI-3200 URL: https://issues.apache.org/jira/browse/HUDI-3200 Project: Apache Hudi

[jira] [Created] (HUDI-3201) Make partition auto discovery configurable

2022-01-09 Thread Raymond Xu (Jira)
Raymond Xu created HUDI-3201: Summary: Make partition auto discovery configurable Key: HUDI-3201 URL: https://issues.apache.org/jira/browse/HUDI-3201 Project: Apache Hudi Issue Type: Improvement

  1   2   >