svn commit: r51125 - in /dev/hudi/hudi-0.10.0-rc2: ./ hudi-0.10.0-rc2.src.tgz hudi-0.10.0-rc2.src.tgz.asc hudi-0.10.0-rc2.src.tgz.sha512

2021-11-29 Thread danny0405
Author: danny0405 Date: Mon Nov 29 08:25:24 2021 New Revision: 51125 Log: Apache Hudi, version 0.10.0, release candidate 2 Added: dev/hudi/hudi-0.10.0-rc2/ dev/hudi/hudi-0.10.0-rc2/hudi-0.10.0-rc2.src.tgz (with props) dev/hudi/hudi-0.10.0-rc2/hudi-0.10.0-rc2.src.tgz.asc dev/hudi

[GitHub] [hudi] zhangyue19921010 opened a new pull request #3887: [HUDI-2648] Retry FileSystem action instead of failed directly.

2021-11-29 Thread GitBox
zhangyue19921010 opened a new pull request #3887: URL: https://github.com/apache/hudi/pull/3887 https://issues.apache.org/jira/browse/HUDI-2648 ## What is the purpose of the pull request Hoodie will do lots of list/get/put/delete etc actions on filesystem. Sometimes will meet th

[GitHub] [hudi] zhangyue19921010 commented on pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

2021-11-29 Thread GitBox
zhangyue19921010 commented on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-981424079 Hi @codope would you like to take a look at this pr? Really appreciate it if you could give me a hand :> -- This is an automated message from the Apache Git Service. To

[GitHub] [hudi] suribabu-un opened a new issue #4151: [SUPPORT] ClassNotFoundException: org.apache.hudi.hadoop.HoodieParquetInputFormat while running hive queries in EMR

2021-11-29 Thread GitBox
suribabu-un opened a new issue #4151: URL: https://github.com/apache/hudi/issues/4151 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://cwiki.apache.org/confluence/display/HUDI/FAQ)? - Join the mailing list to engage in conversations and get faster s

[GitHub] [hudi] xiarixiaoyao commented on pull request #4139: [HUDI-2876] for hive/presto hudi should remove the temp file which cr…

2021-11-29 Thread GitBox
xiarixiaoyao commented on pull request #4139: URL: https://github.com/apache/hudi/pull/4139#issuecomment-981554585 In fact, spark also has this problem, but spark is not resident, so the problem is not obvious, Do you think spark should also be repaired ? -- This is an automated messa

[GitHub] [hudi] YannByron commented on issue #4102: [SUPPORT] What should I do if I want to delete data in certain partitions?

2021-11-29 Thread GitBox
YannByron commented on issue #4102: URL: https://github.com/apache/hudi/issues/4102#issuecomment-981570788 > @YannByron , we can delete partition with above cmd if only we had integrated spark-hive, right ? yep, execute this command on spark, with hudi 0.10. -- This is an automate

[GitHub] [hudi] jasondavindev commented on issue #4122: [SUPPORT] UPDATE command doest not working on Spark SQL

2021-11-29 Thread GitBox
jasondavindev commented on issue #4122: URL: https://github.com/apache/hudi/issues/4122#issuecomment-981610020 @xushiyan Thanks! I built the image, but when I trying write a dataframe, I receive the error ```bash >>> df.write.format('hudi').options(**hudi_options).save('/tmp/data/sam

[GitHub] [hudi] jasondavindev edited a comment on issue #4122: [SUPPORT] UPDATE command doest not working on Spark SQL

2021-11-29 Thread GitBox
jasondavindev edited a comment on issue #4122: URL: https://github.com/apache/hudi/issues/4122#issuecomment-981610020 @xushiyan Thanks! I built the image, but when I trying write a dataframe, I receive the error ```bash >>> df.write.format('hudi').options(**hudi_options).save('/tmp/d

[GitHub] [hudi] jasondavindev edited a comment on issue #4122: [SUPPORT] UPDATE command doest not working on Spark SQL

2021-11-29 Thread GitBox
jasondavindev edited a comment on issue #4122: URL: https://github.com/apache/hudi/issues/4122#issuecomment-981610020 @xushiyan Thanks! I built the image, but when I trying write a dataframe, I receive the error ```bash >>> df.write.format('hudi').options(**hudi_options).save('/tmp/d

[GitHub] [hudi] kenny-shopback commented on issue #4102: [SUPPORT] What should I do if I want to delete data in certain partitions?

2021-11-29 Thread GitBox
kenny-shopback commented on issue #4102: URL: https://github.com/apache/hudi/issues/4102#issuecomment-981650081 thank @xushiyan , @YannByron -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Created] (HUDI-2881) Compact the file group with larger log files to reduce write amplification

2021-11-29 Thread XiaoyuGeng (Jira)
XiaoyuGeng created HUDI-2881: Summary: Compact the file group with larger log files to reduce write amplification Key: HUDI-2881 URL: https://issues.apache.org/jira/browse/HUDI-2881 Project: Apache Hudi

[GitHub] [hudi] minihippo opened a new pull request #4152: [HUDI-2881] Compact the file group with larger log files to reduce wr…

2021-11-29 Thread GitBox
minihippo opened a new pull request #4152: URL: https://github.com/apache/hudi/pull/4152 …ite amplification by using LogFileSizeThresholdBasedCompactionStrategy. ## What is the purpose of the pull request For huge table, it has many file groups and each file group size can be more

[jira] [Updated] (HUDI-2881) Compact the file group with larger log files to reduce write amplification

2021-11-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2881: - Labels: pull-request-available (was: ) > Compact the file group with larger log files to reduce w

[GitHub] [hudi] minihippo commented on pull request #4152: [HUDI-2881] Compact the file group with larger log files to reduce wr…

2021-11-29 Thread GitBox
minihippo commented on pull request #4152: URL: https://github.com/apache/hudi/pull/4152#issuecomment-981704859 @leesf cloud u help me to review this pr? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [hudi] hudi-bot commented on pull request #4152: [HUDI-2881] Compact the file group with larger log files to reduce wr…

2021-11-29 Thread GitBox
hudi-bot commented on pull request #4152: URL: https://github.com/apache/hudi/pull/4152#issuecomment-981706570 ## CI report: * 51e4a9e874a5b1c578ba6508fbf2652714c9626c UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4152: [HUDI-2881] Compact the file group with larger log files to reduce wr…

2021-11-29 Thread GitBox
hudi-bot removed a comment on pull request #4152: URL: https://github.com/apache/hudi/pull/4152#issuecomment-981706570 ## CI report: * 51e4a9e874a5b1c578ba6508fbf2652714c9626c UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4152: [HUDI-2881] Compact the file group with larger log files to reduce wr…

2021-11-29 Thread GitBox
hudi-bot commented on pull request #4152: URL: https://github.com/apache/hudi/pull/4152#issuecomment-981708840 ## CI report: * 51e4a9e874a5b1c578ba6508fbf2652714c9626c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot commented on pull request #4152: [HUDI-2881] Compact the file group with larger log files to reduce wr…

2021-11-29 Thread GitBox
hudi-bot commented on pull request #4152: URL: https://github.com/apache/hudi/pull/4152#issuecomment-981756626 ## CI report: * 51e4a9e874a5b1c578ba6508fbf2652714c9626c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4152: [HUDI-2881] Compact the file group with larger log files to reduce wr…

2021-11-29 Thread GitBox
hudi-bot removed a comment on pull request #4152: URL: https://github.com/apache/hudi/pull/4152#issuecomment-981708840 ## CI report: * 51e4a9e874a5b1c578ba6508fbf2652714c9626c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] lsyldliu opened a new pull request #4153: [HUDI-2877] Support flink catalog to help user use flink table conveniently

2021-11-29 Thread GitBox
lsyldliu opened a new pull request #4153: URL: https://github.com/apache/hudi/pull/4153 …iently ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## Wha

[jira] [Updated] (HUDI-2877) Support flink catalog to help user use flink table conveniently

2021-11-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2877: - Labels: pull-request-available (was: ) > Support flink catalog to help user use flink table conve

[GitHub] [hudi] hudi-bot commented on pull request #4153: [HUDI-2877] Support flink catalog to help user use flink table conveniently

2021-11-29 Thread GitBox
hudi-bot commented on pull request #4153: URL: https://github.com/apache/hudi/pull/4153#issuecomment-981770536 ## CI report: * 4d0c55169a2ff9652bb63e92544d5a89dc9da71f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4153: [HUDI-2877] Support flink catalog to help user use flink table conveniently

2021-11-29 Thread GitBox
hudi-bot removed a comment on pull request #4153: URL: https://github.com/apache/hudi/pull/4153#issuecomment-981770536 ## CI report: * 4d0c55169a2ff9652bb63e92544d5a89dc9da71f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4153: [HUDI-2877] Support flink catalog to help user use flink table conveniently

2021-11-29 Thread GitBox
hudi-bot commented on pull request #4153: URL: https://github.com/apache/hudi/pull/4153#issuecomment-981773016 ## CI report: * 4d0c55169a2ff9652bb63e92544d5a89dc9da71f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot commented on pull request #4153: [HUDI-2877] Support flink catalog to help user use flink table conveniently

2021-11-29 Thread GitBox
hudi-bot commented on pull request #4153: URL: https://github.com/apache/hudi/pull/4153#issuecomment-981826925 ## CI report: * 4d0c55169a2ff9652bb63e92544d5a89dc9da71f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4153: [HUDI-2877] Support flink catalog to help user use flink table conveniently

2021-11-29 Thread GitBox
hudi-bot removed a comment on pull request #4153: URL: https://github.com/apache/hudi/pull/4153#issuecomment-981773016 ## CI report: * 4d0c55169a2ff9652bb63e92544d5a89dc9da71f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[jira] [Assigned] (HUDI-2878) Enhance hudi-quick start guide for spark-sql

2021-11-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-2878: - Assignee: Yann Byron > Enhance hudi-quick start guide for spark-sql > ---

[GitHub] [hudi] BenjMaq opened a new issue #4154: [SUPPORT] INSERT OVERWRITE operation does not work when using Spark SQL

2021-11-29 Thread GitBox
BenjMaq opened a new issue #4154: URL: https://github.com/apache/hudi/issues/4154 **Describe the problem you faced** _Disclaimer: Creating and inserting into external hive tables stored on S3._ - The `INSERT OVERWRITE` operation does not work when using spark SQL. When running

[jira] [Created] (HUDI-2882) Validate staged release script should check and warn for all dependencies before the run

2021-11-29 Thread Manoj Govindassamy (Jira)
Manoj Govindassamy created HUDI-2882: Summary: Validate staged release script should check and warn for all dependencies before the run Key: HUDI-2882 URL: https://issues.apache.org/jira/browse/HUDI-2882

[jira] [Created] (HUDI-2883) Refactor Hive Sync tool /config to use reflection and move to hudi sync common package

2021-11-29 Thread Rajesh Mahindra (Jira)
Rajesh Mahindra created HUDI-2883: - Summary: Refactor Hive Sync tool /config to use reflection and move to hudi sync common package Key: HUDI-2883 URL: https://issues.apache.org/jira/browse/HUDI-2883

[jira] [Commented] (HUDI-2757) Support AWS Glue API for metastore sync

2021-11-29 Thread Rajesh Mahindra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17450655#comment-17450655 ] Rajesh Mahindra commented on HUDI-2757: --- Blocked on HUDI-2883 > Support AWS Glue AP

[GitHub] [hudi] umehrot2 merged pull request #4087: [HUDI-2775] Add doc for external configuration and dynamodb based lock

2021-11-29 Thread GitBox
umehrot2 merged pull request #4087: URL: https://github.com/apache/hudi/pull/4087 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr

[hudi] branch asf-site updated: [HUDI-2775] Add doc for external configuration and dynamodb based lock (#4087)

2021-11-29 Thread uditme
This is an automated email from the ASF dual-hosted git repository. uditme pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 06b0098 [HUDI-2775] Add doc for external conf

[jira] [Created] (HUDI-2884) Allow loading external configs while querying Hudi tables with Spark

2021-11-29 Thread Wenning Ding (Jira)
Wenning Ding created HUDI-2884: -- Summary: Allow loading external configs while querying Hudi tables with Spark Key: HUDI-2884 URL: https://issues.apache.org/jira/browse/HUDI-2884 Project: Apache Hudi

[GitHub] [hudi] zhedoubushishi opened a new pull request #4155: [WIP] [HUDI-2884] Allow loading external configs while querying Hudi tables

2021-11-29 Thread GitBox
zhedoubushishi opened a new pull request #4155: URL: https://github.com/apache/hudi/pull/4155 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

[jira] [Updated] (HUDI-2884) Allow loading external configs while querying Hudi tables with Spark

2021-11-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2884: - Labels: pull-request-available (was: ) > Allow loading external configs while querying Hudi table

[GitHub] [hudi] hudi-bot commented on pull request #4155: [WIP] [HUDI-2884] Allow loading external configs while querying Hudi tables

2021-11-29 Thread GitBox
hudi-bot commented on pull request #4155: URL: https://github.com/apache/hudi/pull/4155#issuecomment-982003462 ## CI report: * 08ac02b55a0641f27ddd2108fce92eec47a0c6c4 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4155: [WIP] [HUDI-2884] Allow loading external configs while querying Hudi tables

2021-11-29 Thread GitBox
hudi-bot removed a comment on pull request #4155: URL: https://github.com/apache/hudi/pull/4155#issuecomment-982003462 ## CI report: * 08ac02b55a0641f27ddd2108fce92eec47a0c6c4 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4155: [WIP] [HUDI-2884] Allow loading external configs while querying Hudi tables

2021-11-29 Thread GitBox
hudi-bot commented on pull request #4155: URL: https://github.com/apache/hudi/pull/4155#issuecomment-982005846 ## CI report: * 08ac02b55a0641f27ddd2108fce92eec47a0c6c4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[jira] [Closed] (HUDI-2831) Make sure usages of any `DateFormat`-based components are thread-safe

2021-11-29 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin closed HUDI-2831. - > Make sure usages of any `DateFormat`-based components are thread-safe >

[jira] [Closed] (HUDI-2788) Z-ordering Layout Optimization Strategy fails w/ Data Skipping enabled

2021-11-29 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin closed HUDI-2788. - > Z-ordering Layout Optimization Strategy fails w/ Data Skipping enabled > ---

[jira] [Updated] (HUDI-2814) Address issues w/ Z-order Layout Optimization

2021-11-29 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-2814: -- Status: Resolved (was: Patch Available) > Address issues w/ Z-order Layout Optimization > -

[jira] [Closed] (HUDI-2840) Configuration from CLI and properties files is not interoperable

2021-11-29 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin closed HUDI-2840. - > Configuration from CLI and properties files is not interoperable > -

[jira] [Closed] (HUDI-2814) Address issues w/ Z-order Layout Optimization

2021-11-29 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin closed HUDI-2814. - > Address issues w/ Z-order Layout Optimization > - > >

[GitHub] [hudi] prashantwason commented on a change in pull request #4020: [WIP][HUDI-2783] Upgrade HBase to 2.x

2021-11-29 Thread GitBox
prashantwason commented on a change in pull request #4020: URL: https://github.com/apache/hudi/pull/4020#discussion_r758727130 ## File path: hudi-common/src/main/java/org/apache/hudi/io/storage/HoodieHFileReader.java ## @@ -72,56 +75,58 @@ public HoodieHFileReader(Configura

[GitHub] [hudi] hudi-bot removed a comment on pull request #4155: [WIP] [HUDI-2884] Allow loading external configs while querying Hudi tables

2021-11-29 Thread GitBox
hudi-bot removed a comment on pull request #4155: URL: https://github.com/apache/hudi/pull/4155#issuecomment-982005846 ## CI report: * 08ac02b55a0641f27ddd2108fce92eec47a0c6c4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4155: [WIP] [HUDI-2884] Allow loading external configs while querying Hudi tables

2021-11-29 Thread GitBox
hudi-bot commented on pull request #4155: URL: https://github.com/apache/hudi/pull/4155#issuecomment-982042551 ## CI report: * 08ac02b55a0641f27ddd2108fce92eec47a0c6c4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] prashantwason commented on a change in pull request #3590: [HUDI-2285][HUDI-2476] Metadata table synchronous design. Rebased and Squashed from pull/3426

2021-11-29 Thread GitBox
prashantwason commented on a change in pull request #3590: URL: https://github.com/apache/hudi/pull/3590#discussion_r758755681 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTimelineArchiveLog.java ## @@ -200,20 +200,19 @@ public boolean

[GitHub] [hudi] rmahindra123 opened a new pull request #4157: Rm add banner

2021-11-29 Thread GitBox
rmahindra123 opened a new pull request #4157: URL: https://github.com/apache/hudi/pull/4157 Add a banner to call for voting for release 0.10.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] nsivabalan opened a new pull request #4156: [MINOR] Adding halodoc link to list of articles

2021-11-29 Thread GitBox
nsivabalan opened a new pull request #4156: URL: https://github.com/apache/hudi/pull/4156 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purp

[jira] [Created] (HUDI-2885) Add call for voting for latest release on hudi homepage

2021-11-29 Thread Rajesh Mahindra (Jira)
Rajesh Mahindra created HUDI-2885: - Summary: Add call for voting for latest release on hudi homepage Key: HUDI-2885 URL: https://issues.apache.org/jira/browse/HUDI-2885 Project: Apache Hudi I

[hudi] branch asf-site updated: [MINOR] Adding halodoc article link to website (#4156)

2021-11-29 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new c2dff10 [MINOR] Adding halodoc article lin

[GitHub] [hudi] nsivabalan merged pull request #4156: [MINOR] Adding halodoc link to list of articles

2021-11-29 Thread GitBox
nsivabalan merged pull request #4156: URL: https://github.com/apache/hudi/pull/4156 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[GitHub] [hudi] bhasudha merged pull request #4157: [HUDI-2885] Add banner to call for voting for latest release version

2021-11-29 Thread GitBox
bhasudha merged pull request #4157: URL: https://github.com/apache/hudi/pull/4157 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr

[hudi] branch asf-site updated: Remove stale commits (#4157)

2021-11-29 Thread bhavanisudha
This is an automated email from the ASF dual-hosted git repository. bhavanisudha pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new f268709 Remove stale commits (#4157) f2

[jira] [Updated] (HUDI-2885) Add call for voting for latest release on hudi homepage

2021-11-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2885: - Labels: pull-request-available (was: ) > Add call for voting for latest release on hudi homepage

[GitHub] [hudi] prashantwason commented on a change in pull request #3865: [HUDI-2005] Removing direct fs call in HoodieLogFileReader

2021-11-29 Thread GitBox
prashantwason commented on a change in pull request #3865: URL: https://github.com/apache/hudi/pull/3865#discussion_r758795588 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/LogReaderUtils.java ## @@ -27,24 +27,26 @@ import org.apache.hudi.common.ta

[GitHub] [hudi] rmahindra123 opened a new pull request #4158: [HUDI-2885] Fix the latest release banner

2021-11-29 Thread GitBox
rmahindra123 opened a new pull request #4158: URL: https://github.com/apache/hudi/pull/4158 [HUDI-2885] Fix the latest release banner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] rmahindra123 closed pull request #4158: [HUDI-2885] Fix the latest release banner

2021-11-29 Thread GitBox
rmahindra123 closed pull request #4158: URL: https://github.com/apache/hudi/pull/4158 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsu

[jira] [Commented] (HUDI-2478) Handle failure mid-way during init buckets

2021-11-29 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17450759#comment-17450759 ] Prashant Wason commented on HUDI-2478: -- The recovery today [is due to this code|http

[GitHub] [hudi] rmahindra123 opened a new pull request #4159: [HUDI-2885] Fix the latest release banner

2021-11-29 Thread GitBox
rmahindra123 opened a new pull request #4159: URL: https://github.com/apache/hudi/pull/4159 [HUDI-2885] Fix the latest release banner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [hudi] bhasudha merged pull request #4159: [HUDI-2885] Fix the latest release banner

2021-11-29 Thread GitBox
bhasudha merged pull request #4159: URL: https://github.com/apache/hudi/pull/4159 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr

[hudi] branch asf-site updated: Fix version number (#4159)

2021-11-29 Thread bhavanisudha
This is an automated email from the ASF dual-hosted git repository. bhavanisudha pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 504dc3a Fix version number (#4159) 504d

[GitHub] [hudi] rmahindra123 commented on a change in pull request #4107: [HUDI-2825] - Docs for markers

2021-11-29 Thread GitBox
rmahindra123 commented on a change in pull request #4107: URL: https://github.com/apache/hudi/pull/4107#discussion_r758806948 ## File path: website/docs/configurations.md ## @@ -1151,8 +1151,8 @@ Configurations that control write behavior on Hudi tables. These can be directly

[jira] [Commented] (HUDI-2458) Relax compaction in metadata being fenced based on inflight requests in data table

2021-11-29 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17450764#comment-17450764 ] Prashant Wason commented on HUDI-2458: -- >> Archival in data table does not have any d

[GitHub] [hudi] zhedoubushishi closed pull request #4155: [WIP] [HUDI-2884] Allow loading external configs while querying Hudi tables

2021-11-29 Thread GitBox
zhedoubushishi closed pull request #4155: URL: https://github.com/apache/hudi/pull/4155 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-un

[jira] [Commented] (HUDI-2458) Relax compaction in metadata being fenced based on inflight requests in data table

2021-11-29 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17450766#comment-17450766 ] Prashant Wason commented on HUDI-2458: -- Another way to solve this issue is to think i

[GitHub] [hudi] yihua commented on pull request #4116: [HUDI-2856] Bit cask disk map delete modified

2021-11-29 Thread GitBox
yihua commented on pull request #4116: URL: https://github.com/apache/hudi/pull/4116#issuecomment-982139242 When running the Hudi Kafka Connect Sink, although the writes are successful, the sink keeps throwing the following repetitive errors, which can be noisy to users. After reverting t

[GitHub] [hudi] hudi-bot commented on pull request #4106: [HUDI-2814] Make Z-index more generic Column-Stats Index

2021-11-29 Thread GitBox
hudi-bot commented on pull request #4106: URL: https://github.com/apache/hudi/pull/4106#issuecomment-982165274 ## CI report: * 9faa878daccde5bda6cc1e3f0b750b160037b9ee Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4106: [HUDI-2814] Make Z-index more generic Column-Stats Index

2021-11-29 Thread GitBox
hudi-bot removed a comment on pull request #4106: URL: https://github.com/apache/hudi/pull/4106#issuecomment-978921317 ## CI report: * 9faa878daccde5bda6cc1e3f0b750b160037b9ee Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4106: [HUDI-2814] Make Z-index more generic Column-Stats Index

2021-11-29 Thread GitBox
hudi-bot commented on pull request #4106: URL: https://github.com/apache/hudi/pull/4106#issuecomment-982167169 ## CI report: * 9faa878daccde5bda6cc1e3f0b750b160037b9ee Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4106: [HUDI-2814] Make Z-index more generic Column-Stats Index

2021-11-29 Thread GitBox
hudi-bot removed a comment on pull request #4106: URL: https://github.com/apache/hudi/pull/4106#issuecomment-982165274 ## CI report: * 9faa878daccde5bda6cc1e3f0b750b160037b9ee Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot removed a comment on pull request #4106: [HUDI-2814] Make Z-index more generic Column-Stats Index

2021-11-29 Thread GitBox
hudi-bot removed a comment on pull request #4106: URL: https://github.com/apache/hudi/pull/4106#issuecomment-982167169 ## CI report: * 9faa878daccde5bda6cc1e3f0b750b160037b9ee Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4106: [HUDI-2814] Make Z-index more generic Column-Stats Index

2021-11-29 Thread GitBox
hudi-bot commented on pull request #4106: URL: https://github.com/apache/hudi/pull/4106#issuecomment-982198136 ## CI report: * 156d91843297f44acd838afe8da3439daa5b7a0e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] mincwang opened a new issue #4160: hive and spark query column result are different

2021-11-29 Thread GitBox
mincwang opened a new issue #4160: URL: https://github.com/apache/hudi/issues/4160 This patch is great, but there is one small problem,following: - when I delete record with id 5 hive query reslut: ![image](https://user-images.githubusercontent.com/33626973/143387325-dadd2ba0-d0fa

[GitHub] [hudi] mincwang commented on issue #4160: hive and spark query column result are different

2021-11-29 Thread GitBox
mincwang commented on issue #4160: URL: https://github.com/apache/hudi/issues/4160#issuecomment-982216679 spark reslut lost the `_hoodie_operation` column -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [hudi] mincwang edited a comment on issue #4160: hive and spark query column result are different

2021-11-29 Thread GitBox
mincwang edited a comment on issue #4160: URL: https://github.com/apache/hudi/issues/4160#issuecomment-982216679 ```sql hive> desc user_rt; OK _hoodie_commit_time string _hoodie_commit_seqnostring _hoodie_record_key string _hoodie_partition_path string _hoo

[GitHub] [hudi] mincwang commented on issue #4160: hive and spark query column result are different

2021-11-29 Thread GitBox
mincwang commented on issue #4160: URL: https://github.com/apache/hudi/issues/4160#issuecomment-982217203 cc @garyli1019 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [hudi] xushiyan merged pull request #4130: [MINOR] Fix typo,rename 'getUrlEncodePartitoning' to 'getUrlEncodePartitioning'

2021-11-29 Thread GitBox
xushiyan merged pull request #4130: URL: https://github.com/apache/hudi/pull/4130 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr

[hudi] branch master updated (536af4b -> 3433f00)

2021-11-29 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 536af4b [MINOR] Fix syntax error in create_source_release.sh (#4150) add 3433f00 [MINOR] Fix typo,rename 'getU

[jira] [Created] (HUDI-2886) Certify metadata table using large-scale cluster testing

2021-11-29 Thread Rajesh Mahindra (Jira)
Rajesh Mahindra created HUDI-2886: - Summary: Certify metadata table using large-scale cluster testing Key: HUDI-2886 URL: https://issues.apache.org/jira/browse/HUDI-2886 Project: Apache Hudi

[jira] [Updated] (HUDI-2886) Certify metadata table using large-scale cluster testing

2021-11-29 Thread Rajesh Mahindra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Mahindra updated HUDI-2886: -- Priority: Blocker (was: Major) > Certify metadata table using large-scale cluster testing > ---

[jira] [Created] (HUDI-2887) Certify metadata table (whitebox) testing locally

2021-11-29 Thread Rajesh Mahindra (Jira)
Rajesh Mahindra created HUDI-2887: - Summary: Certify metadata table (whitebox) testing locally Key: HUDI-2887 URL: https://issues.apache.org/jira/browse/HUDI-2887 Project: Apache Hudi Issue T

[jira] [Created] (HUDI-2888) Certify z-order/ hilbert curve using large scale cluster testing

2021-11-29 Thread Rajesh Mahindra (Jira)
Rajesh Mahindra created HUDI-2888: - Summary: Certify z-order/ hilbert curve using large scale cluster testing Key: HUDI-2888 URL: https://issues.apache.org/jira/browse/HUDI-2888 Project: Apache Hudi

[jira] [Created] (HUDI-2890) Certify kafka connect functionality with compaction /clustering

2021-11-29 Thread Rajesh Mahindra (Jira)
Rajesh Mahindra created HUDI-2890: - Summary: Certify kafka connect functionality with compaction /clustering Key: HUDI-2890 URL: https://issues.apache.org/jira/browse/HUDI-2890 Project: Apache Hudi

[jira] [Created] (HUDI-2889) Certify Timeline server: covering all scenarios

2021-11-29 Thread Rajesh Mahindra (Jira)
Rajesh Mahindra created HUDI-2889: - Summary: Certify Timeline server: covering all scenarios Key: HUDI-2889 URL: https://issues.apache.org/jira/browse/HUDI-2889 Project: Apache Hudi Issue Typ

[jira] [Updated] (HUDI-2890) Certify kafka connect functionality with compaction /clustering

2021-11-29 Thread Rajesh Mahindra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Mahindra updated HUDI-2890: -- Fix Version/s: 0.10.0 > Certify kafka connect functionality with compaction /clustering > --

[GitHub] [hudi] YannByron commented on a change in pull request #4138: [HUDI-2781] Set spark3 in azure pipelines

2021-11-29 Thread GitBox
YannByron commented on a change in pull request #4138: URL: https://github.com/apache/hudi/pull/4138#discussion_r758896163 ## File path: pom.xml ## @@ -1511,7 +1511,7 @@ spark3.0.x -3.0.0 +3.0.3 ${spark3.version} 3.0.1 Review

[jira] [Updated] (HUDI-2891) Kafka-connect sink still uses timeline-server-based markers as default

2021-11-29 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-2891: Priority: Blocker (was: Major) > Kafka-connect sink still uses timeline-server-based markers as default > -

[jira] [Created] (HUDI-2891) Kafka-connect sink still uses timeline-server-based markers as default

2021-11-29 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-2891: --- Summary: Kafka-connect sink still uses timeline-server-based markers as default Key: HUDI-2891 URL: https://issues.apache.org/jira/browse/HUDI-2891 Project: Apache Hudi

[jira] [Updated] (HUDI-2891) Kafka-connect sink still uses timeline-server-based markers as default

2021-11-29 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-2891: Fix Version/s: 0.10.0 > Kafka-connect sink still uses timeline-server-based markers as default > ---

[jira] [Updated] (HUDI-2891) Kafka-connect sink still uses timeline-server-based markers as default

2021-11-29 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-2891: Description: By default, Kafka-connect sink using Java client should use direct markers by default.  Errors

[GitHub] [hudi] danny0405 commented on a change in pull request #3590: [HUDI-2285][HUDI-2476] Metadata table synchronous design. Rebased and Squashed from pull/3426

2021-11-29 Thread GitBox
danny0405 commented on a change in pull request #3590: URL: https://github.com/apache/hudi/pull/3590#discussion_r758909756 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTimelineArchiveLog.java ## @@ -200,20 +200,19 @@ public boolean arc

[GitHub] [hudi] yihua opened a new pull request #4161: [HUDI-2891] Fix write configs for Java engine in Kafka Connect Sink

2021-11-29 Thread GitBox
yihua opened a new pull request #4161: URL: https://github.com/apache/hudi/pull/4161 ## What is the purpose of the pull request This PR fixes the bug where the Hudi Kafka Connect Sink does not respect Java-engine-specific configs, particularly the marker type. By default, Kafka Con

[jira] [Updated] (HUDI-2891) Kafka-connect sink still uses timeline-server-based markers as default

2021-11-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2891: - Labels: pull-request-available (was: ) > Kafka-connect sink still uses timeline-server-based mark

[GitHub] [hudi] hudi-bot commented on pull request #4161: [HUDI-2891] Fix write configs for Java engine in Kafka Connect Sink

2021-11-29 Thread GitBox
hudi-bot commented on pull request #4161: URL: https://github.com/apache/hudi/pull/4161#issuecomment-982260395 ## CI report: * d25ed00fdbceb70376ba3a01170ede389367211c UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4161: [HUDI-2891] Fix write configs for Java engine in Kafka Connect Sink

2021-11-29 Thread GitBox
hudi-bot removed a comment on pull request #4161: URL: https://github.com/apache/hudi/pull/4161#issuecomment-982260395 ## CI report: * d25ed00fdbceb70376ba3a01170ede389367211c UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4161: [HUDI-2891] Fix write configs for Java engine in Kafka Connect Sink

2021-11-29 Thread GitBox
hudi-bot commented on pull request #4161: URL: https://github.com/apache/hudi/pull/4161#issuecomment-982261183 ## CI report: * d25ed00fdbceb70376ba3a01170ede389367211c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot commented on pull request #4161: [HUDI-2891] Fix write configs for Java engine in Kafka Connect Sink

2021-11-29 Thread GitBox
hudi-bot commented on pull request #4161: URL: https://github.com/apache/hudi/pull/4161#issuecomment-982264832 ## CI report: * d25ed00fdbceb70376ba3a01170ede389367211c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4161: [HUDI-2891] Fix write configs for Java engine in Kafka Connect Sink

2021-11-29 Thread GitBox
hudi-bot removed a comment on pull request #4161: URL: https://github.com/apache/hudi/pull/4161#issuecomment-982261183 ## CI report: * d25ed00fdbceb70376ba3a01170ede389367211c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

  1   2   >