[jira] [Commented] (HUDI-2705) Metadata based column stats index - PoC

2022-01-12 Thread Manoj Govindassamy (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17474338#comment-17474338 ] Manoj Govindassamy commented on HUDI-2705: -- [~shibei] Please take a look [https:/

[GitHub] [hudi] hudi-bot commented on pull request #4569: [HUDI-3225] Claim RFC-45 for async metadata indexing

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4569: URL: https://github.com/apache/hudi/pull/4569#issuecomment-1010749510 ## CI report: * fc437f3c4f72f7d02078b3d887578d7b010cca01 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4569: [HUDI-3225] Claim RFC-45 for async metadata indexing

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4569: URL: https://github.com/apache/hudi/pull/4569#issuecomment-1010713141 ## CI report: * fc437f3c4f72f7d02078b3d887578d7b010cca01 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4566: [HUDI-3218] Upgrading avro to 1.10.2

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4566: URL: https://github.com/apache/hudi/pull/4566#issuecomment-1010665129 ## CI report: * 7ddc515ec17aa7de9585bfc30ba638ca18f89ddf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4566: [HUDI-3218] Upgrading avro to 1.10.2

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4566: URL: https://github.com/apache/hudi/pull/4566#issuecomment-1010757343 ## CI report: * b5e9f453755f5541362fa70f2c37c408c8f8bdd9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4083: [HUDI-2837] Add support for using database name in incremental query

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1010723861 ## CI report: * 00221c82e8b1693280fd72625eafcd503d54323c UNKNOWN * 46053bb143d1fd1274ac466197cc9361708e738b UNKNOWN * 2722bfcfd29a95f27338c1c8b026185472eef

[GitHub] [hudi] hudi-bot commented on pull request #4083: [HUDI-2837] Add support for using database name in incremental query

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1010762027 ## CI report: * 00221c82e8b1693280fd72625eafcd503d54323c UNKNOWN * 46053bb143d1fd1274ac466197cc9361708e738b UNKNOWN * 2722bfcfd29a95f27338c1c8b026185472eefba0 UNKN

[GitHub] [hudi] hudi-bot commented on pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4514: URL: https://github.com/apache/hudi/pull/4514#issuecomment-1010762438 ## CI report: * ddc3af0c32bafef6b10c32c43132df32a5f7d83c UNKNOWN * e1ba726105dfa7ae07d802546c71a0cf1ad8b172 UNKNOWN * 306e7d462959e0249e230f60c2e9ea6602342e08 UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4514: URL: https://github.com/apache/hudi/pull/4514#issuecomment-1010744877 ## CI report: * ddc3af0c32bafef6b10c32c43132df32a5f7d83c UNKNOWN * e1ba726105dfa7ae07d802546c71a0cf1ad8b172 UNKNOWN * 306e7d462959e0249e230f60c2e9ea6602342

[jira] [Comment Edited] (HUDI-3204) spark on TimestampBasedKeyGenerator has no result when query by partition column

2022-01-12 Thread taisenki (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17472398#comment-17472398 ] taisenki edited comment on HUDI-3204 at 1/12/22, 8:26 AM: -- [~biya

[GitHub] [hudi] hudi-bot removed a comment on pull request #4564: [HUDI-3007] Fix issues in HoodieRepairTool

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4564: URL: https://github.com/apache/hudi/pull/4564#issuecomment-1010685205 ## CI report: * 92bcb6959aca76c9945f36956c0514e1f1440659 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4564: [HUDI-3007] Fix issues in HoodieRepairTool

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4564: URL: https://github.com/apache/hudi/pull/4564#issuecomment-1010774701 ## CI report: * 3369d965c670a2ee0fdb1deaf60b6bfcaf35624a Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4300: [HUDI-2785] Add Trino setup in Docker Demo

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4300: URL: https://github.com/apache/hudi/pull/4300#issuecomment-1010676346 ## CI report: * aa03fbdf8ecef97f82ab9acbcf5b227a45b785aa Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4300: [HUDI-2785] Add Trino setup in Docker Demo

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4300: URL: https://github.com/apache/hudi/pull/4300#issuecomment-1010784995 ## CI report: * 28c1a0469f575bfd0bbe30b71460edb80d0e0517 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[jira] [Created] (HUDI-3231) flink-hudi增量查询，时间点查询

2022-01-12 Thread Jira
席宇鹏 created HUDI-3231: - Summary: flink-hudi增量查询，时间点查询 Key: HUDI-3231 URL: https://issues.apache.org/jira/browse/HUDI-3231 Project: Apache Hudi Issue Type: Test Reporter: 席宇鹏 请问一下，hudi0.10的f

[jira] [Updated] (HUDI-3231) flink-hudi增量查询，时间点查询

2022-01-12 Thread Jira
[ https://issues.apache.org/jira/browse/HUDI-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 席宇鹏 updated HUDI-3231: -- Description: 请问一下，hudi0.10的flink增量查询、时间点查询有问题没有，我这测了总感觉输出的结果不对。我指明read.start-commit、read.end-commit两个参数，这期间修改过的同一id的数据不能全

[GitHub] [hudi] danny0405 opened a new pull request #4571: [HUDI-3230] Add streaming read for flink document

2022-01-12 Thread GitBox
danny0405 opened a new pull request #4571: URL: https://github.com/apache/hudi/pull/4571 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpo

[jira] [Updated] (HUDI-3230) Add streaming read for flink document

2022-01-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3230: - Labels: pull-request-available (was: ) > Add streaming read for flink document >

[GitHub] [hudi] YuweiXiao commented on pull request #4346: [HUDI-3045] New clustering regex match config to choose partitions when building clustering plan

2022-01-12 Thread GitBox
YuweiXiao commented on pull request #4346: URL: https://github.com/apache/hudi/pull/4346#issuecomment-1010801226 @yihua I feel it would be better to add a new option in `ClusteringPlanPartitionFilterMode` rather than doing regex in place. -- This is an automated message from the Apache G

[GitHub] [hudi] danny0405 merged pull request #4571: [HUDI-3230] Add streaming read for flink document

2022-01-12 Thread GitBox
danny0405 merged pull request #4571: URL: https://github.com/apache/hudi/pull/4571 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[jira] [Commented] (HUDI-3230) Add streaming read for flink document

2022-01-12 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17474383#comment-17474383 ] Danny Chen commented on HUDI-3230: -- Fixed via master branch: 7374326ccc160367c566e7052eab

[hudi] branch asf-site updated: [HUDI-3230] Add streaming read for flink document (#4571)

2022-01-12 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 7374326 [HUDI-3230] Add streaming read for

[jira] [Resolved] (HUDI-3230) Add streaming read for flink document

2022-01-12 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-3230. -- > Add streaming read for flink document > - > > Key: HUD

[jira] [Updated] (HUDI-3230) Add streaming read for flink document

2022-01-12 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-3230: - Fix Version/s: 0.10.1 (was: 0.10.0) > Add streaming read for flink document > -

[jira] [Updated] (HUDI-2968) Support Delete/Update using non-pk fields

2022-01-12 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2968: - Status: In Progress (was: Open) > Support Delete/Update using non-pk fields > ---

[GitHub] [hudi] hudi-bot removed a comment on pull request #4561: handleEndInputEvent is executed synchronously

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4561: URL: https://github.com/apache/hudi/pull/4561#issuecomment-1010728262 ## CI report: * 5d62bd2fa92c39eba896a3f228806d51f83c4942 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4561: handleEndInputEvent is executed synchronously

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4561: URL: https://github.com/apache/hudi/pull/4561#issuecomment-1010828523 ## CI report: * c7c1430c47702854adc694486ded47b47dfb5a52 UNKNOWN * 3dd749392fc492265a3fe0af3dd0a0d4f3a9cc86 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org

[jira] [Created] (HUDI-3232) support reload timeline Incrementally

2022-01-12 Thread Yann Byron (Jira)
Yann Byron created HUDI-3232: Summary: support reload timeline Incrementally Key: HUDI-3232 URL: https://issues.apache.org/jira/browse/HUDI-3232 Project: Apache Hudi Issue Type: Improvement

[jira] [Commented] (HUDI-1558) Struct Stream Source Support Spark3

2022-01-12 Thread Hui An (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17474395#comment-17474395 ] Hui An commented on HUDI-1558: -- Hi, [~pzw2018], Any updates on this ticket? I'm willing to so

[GitHub] [hudi] hudi-bot commented on pull request #4180: [HUDI-2903] get table schema from the last commit with data written

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4180: URL: https://github.com/apache/hudi/pull/4180#issuecomment-1010849850 ## CI report: * 5ad7241fbe98875c71a3aa4d394cd95f266ae5d9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4180: [HUDI-2903] get table schema from the last commit with data written

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4180: URL: https://github.com/apache/hudi/pull/4180#issuecomment-1010736728 ## CI report: * 74699cc12fa641613dfbce4f05f7cc6b08ed631f Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] pratyakshsharma commented on a change in pull request #3929: [HUDI-1881] Make multi table delta streamer to use thread pool for table sync asynchronously.

2022-01-12 Thread GitBox
pratyakshsharma commented on a change in pull request #3929: URL: https://github.com/apache/hudi/pull/3929#discussion_r782921008 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieMultiTableDeltaStreamer.java ## @@ -378,16 +383,23 @@ priva

[GitHub] [hudi] hudi-bot commented on pull request #4561: handleEndInputEvent is executed synchronously

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4561: URL: https://github.com/apache/hudi/pull/4561#issuecomment-1010884567 ## CI report: * c7c1430c47702854adc694486ded47b47dfb5a52 UNKNOWN * 3dd749392fc492265a3fe0af3dd0a0d4f3a9cc86 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot commented on pull request #4568: [HUDI-2968] add UT for update/delete on non-pk condition

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4568: URL: https://github.com/apache/hudi/pull/4568#issuecomment-1010884639 ## CI report: * 4ed25dc4a2b38ae69934cdfd521d56b2ed60e331 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4561: handleEndInputEvent is executed synchronously

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4561: URL: https://github.com/apache/hudi/pull/4561#issuecomment-1010828523 ## CI report: * c7c1430c47702854adc694486ded47b47dfb5a52 UNKNOWN * 3dd749392fc492265a3fe0af3dd0a0d4f3a9cc86 Azure: [SUCCESS](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot removed a comment on pull request #4568: [HUDI-2968] add UT for update/delete on non-pk condition

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4568: URL: https://github.com/apache/hudi/pull/4568#issuecomment-1010742987 ## CI report: * a42de8afbb24a4d7e5aa821d1a7fc689b6780735 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] pratyakshsharma commented on a change in pull request #4385: [HUDI-1436]: provided option to trigger clean every nth commit

2022-01-12 Thread GitBox
pratyakshsharma commented on a change in pull request #4385: URL: https://github.com/apache/hudi/pull/4385#discussion_r782927390 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieCompactionConfig.java ## @@ -546,11 +568,6 @@ public Builder

[GitHub] [hudi] pratyakshsharma commented on a change in pull request #4385: [HUDI-1436]: provided option to trigger clean every nth commit

2022-01-12 Thread GitBox
pratyakshsharma commented on a change in pull request #4385: URL: https://github.com/apache/hudi/pull/4385#discussion_r782933953 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanActionExecutor.java ## @@ -58,8 +59,34 @@ pub

[GitHub] [hudi] pratyakshsharma commented on a change in pull request #4385: [HUDI-1436]: provided option to trigger clean every nth commit

2022-01-12 Thread GitBox
pratyakshsharma commented on a change in pull request #4385: URL: https://github.com/apache/hudi/pull/4385#discussion_r782934576 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanActionExecutor.java ## @@ -58,8 +59,34 @@ pub

[GitHub] [hudi] hudi-bot commented on pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4514: URL: https://github.com/apache/hudi/pull/4514#issuecomment-1010898171 ## CI report: * ddc3af0c32bafef6b10c32c43132df32a5f7d83c UNKNOWN * e1ba726105dfa7ae07d802546c71a0cf1ad8b172 UNKNOWN * 306e7d462959e0249e230f60c2e9ea6602342e08 UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4514: URL: https://github.com/apache/hudi/pull/4514#issuecomment-1010762438 ## CI report: * ddc3af0c32bafef6b10c32c43132df32a5f7d83c UNKNOWN * e1ba726105dfa7ae07d802546c71a0cf1ad8b172 UNKNOWN * 306e7d462959e0249e230f60c2e9ea6602342

[GitHub] [hudi] hudi-bot commented on pull request #4561: handleEndInputEvent is executed synchronously

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4561: URL: https://github.com/apache/hudi/pull/4561#issuecomment-1010898312 ## CI report: * c7c1430c47702854adc694486ded47b47dfb5a52 UNKNOWN * 3dd749392fc492265a3fe0af3dd0a0d4f3a9cc86 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4561: handleEndInputEvent is executed synchronously

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4561: URL: https://github.com/apache/hudi/pull/4561#issuecomment-1010884567 ## CI report: * c7c1430c47702854adc694486ded47b47dfb5a52 UNKNOWN * 3dd749392fc492265a3fe0af3dd0a0d4f3a9cc86 Azure: [SUCCESS](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot commented on pull request #4083: [HUDI-2837] Add support for using database name in incremental query

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1010903142 ## CI report: * 00221c82e8b1693280fd72625eafcd503d54323c UNKNOWN * 46053bb143d1fd1274ac466197cc9361708e738b UNKNOWN * 2722bfcfd29a95f27338c1c8b026185472eefba0 UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4083: [HUDI-2837] Add support for using database name in incremental query

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1010762027 ## CI report: * 00221c82e8b1693280fd72625eafcd503d54323c UNKNOWN * 46053bb143d1fd1274ac466197cc9361708e738b UNKNOWN * 2722bfcfd29a95f27338c1c8b026185472eef

[jira] [Created] (HUDI-3233) Make metadata commit synchronous for flink batch

2022-01-12 Thread Danny Chen (Jira)
Danny Chen created HUDI-3233: Summary: Make metadata commit synchronous for flink batch Key: HUDI-3233 URL: https://issues.apache.org/jira/browse/HUDI-3233 Project: Apache Hudi Issue Type: Task

[GitHub] [hudi] leesf commented on pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-12 Thread GitBox
leesf commented on pull request #4514: URL: https://github.com/apache/hudi/pull/4514#issuecomment-1010930395 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[GitHub] [hudi] hudi-bot commented on pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4514: URL: https://github.com/apache/hudi/pull/4514#issuecomment-1010932080 ## CI report: * ddc3af0c32bafef6b10c32c43132df32a5f7d83c UNKNOWN * e1ba726105dfa7ae07d802546c71a0cf1ad8b172 UNKNOWN * 306e7d462959e0249e230f60c2e9ea6602342e08 UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4514: URL: https://github.com/apache/hudi/pull/4514#issuecomment-1010898171 ## CI report: * ddc3af0c32bafef6b10c32c43132df32a5f7d83c UNKNOWN * e1ba726105dfa7ae07d802546c71a0cf1ad8b172 UNKNOWN * 306e7d462959e0249e230f60c2e9ea6602342

[GitHub] [hudi] hudi-bot removed a comment on pull request #4561: handleEndInputEvent is executed synchronously

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4561: URL: https://github.com/apache/hudi/pull/4561#issuecomment-1010898312 ## CI report: * c7c1430c47702854adc694486ded47b47dfb5a52 UNKNOWN * 3dd749392fc492265a3fe0af3dd0a0d4f3a9cc86 Azure: [SUCCESS](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot commented on pull request #4561: handleEndInputEvent is executed synchronously

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4561: URL: https://github.com/apache/hudi/pull/4561#issuecomment-1010946913 ## CI report: * c7c1430c47702854adc694486ded47b47dfb5a52 UNKNOWN * 673640240f1a0647526c12f3f17ca565a718af0a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] dongkelun commented on pull request #4083: [HUDI-2837] Add support for using database name in incremental query

2022-01-12 Thread GitBox
dongkelun commented on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1010957136 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot removed a comment on pull request #4083: [HUDI-2837] Add support for using database name in incremental query

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1010903142 ## CI report: * 00221c82e8b1693280fd72625eafcd503d54323c UNKNOWN * 46053bb143d1fd1274ac466197cc9361708e738b UNKNOWN * 2722bfcfd29a95f27338c1c8b026185472eef

[GitHub] [hudi] hudi-bot commented on pull request #4083: [HUDI-2837] Add support for using database name in incremental query

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1010958771 ## CI report: * 00221c82e8b1693280fd72625eafcd503d54323c UNKNOWN * 46053bb143d1fd1274ac466197cc9361708e738b UNKNOWN * 2722bfcfd29a95f27338c1c8b026185472eefba0 UNKN

[GitHub] [hudi] leesf commented on a change in pull request #4350: [HUDI-3047] Basic Implementation of Spark Datasource V2

2022-01-12 Thread GitBox
leesf commented on a change in pull request #4350: URL: https://github.com/apache/hudi/pull/4350#discussion_r783024554 ## File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/hudi/SparkAdapter.scala ## @@ -92,4 +95,31 @@ trait SparkAdapter extends Seria

[GitHub] [hudi] hudi-bot commented on pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4514: URL: https://github.com/apache/hudi/pull/4514#issuecomment-1010996737 ## CI report: * ddc3af0c32bafef6b10c32c43132df32a5f7d83c UNKNOWN * e1ba726105dfa7ae07d802546c71a0cf1ad8b172 UNKNOWN * 306e7d462959e0249e230f60c2e9ea6602342e08 UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4514: URL: https://github.com/apache/hudi/pull/4514#issuecomment-1010932080 ## CI report: * ddc3af0c32bafef6b10c32c43132df32a5f7d83c UNKNOWN * e1ba726105dfa7ae07d802546c71a0cf1ad8b172 UNKNOWN * 306e7d462959e0249e230f60c2e9ea6602342

[GitHub] [hudi] danny0405 closed pull request #4561: [HUDI-3233] Make metadata commit synchronous for flink batch

2022-01-12 Thread GitBox
danny0405 closed pull request #4561: URL: https://github.com/apache/hudi/pull/4561 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[hudi] branch master updated (9fe28e5 -> 2969fb3)

2022-01-12 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 9fe28e5 [HUDI-3045] New clustering regex match config to choose partitions when building clustering plan (#4346)

[jira] [Updated] (HUDI-3233) Make metadata commit synchronous for flink batch

2022-01-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3233: - Labels: pull-request-available (was: ) > Make metadata commit synchronous for flink batch > -

[jira] [Resolved] (HUDI-3233) Make metadata commit synchronous for flink batch

2022-01-12 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-3233. -- > Make metadata commit synchronous for flink batch > > >

[jira] [Commented] (HUDI-3233) Make metadata commit synchronous for flink batch

2022-01-12 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17474487#comment-17474487 ] Danny Chen commented on HUDI-3233: -- Fixed via master branch: 2969fb3835b96dbd31fdbca536e8

[jira] [Assigned] (HUDI-1850) Read on table fails if the first write to table failed

2022-01-12 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-1850: - Assignee: sivabalan narayanan > Read on table fails if the first write to table f

[jira] [Updated] (HUDI-1850) Read on table fails if the first write to table failed

2022-01-12 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-1850: -- Labels: core-flow-ds pull-request-available sev:high spark (was: core-flow-ds pull-requ

[jira] [Updated] (HUDI-1850) Read on table fails if the first write to table failed

2022-01-12 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-1850: -- Status: Resolved (was: Patch Available) > Read on table fails if the first write to tab

[jira] [Reopened] (HUDI-1850) Read on table fails if the first write to table failed

2022-01-12 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reopened HUDI-1850: --- > Read on table fails if the first write to table failed > ---

[jira] [Created] (HUDI-3234) Fixing read of an empty table (with first commit failed) should return empty RDD

2022-01-12 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-3234: - Summary: Fixing read of an empty table (with first commit failed) should return empty RDD Key: HUDI-3234 URL: https://issues.apache.org/jira/browse/HUDI-3234

[jira] [Updated] (HUDI-3234) Fixing read of an empty table (with first commit failed) should return empty RDD

2022-01-12 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3234: -- Description: When first commit fails, read of this table should return empty RDD. Pleas

[GitHub] [hudi] hudi-bot removed a comment on pull request #4083: [HUDI-2837] Add support for using database name in incremental query

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1010958771 ## CI report: * 00221c82e8b1693280fd72625eafcd503d54323c UNKNOWN * 46053bb143d1fd1274ac466197cc9361708e738b UNKNOWN * 2722bfcfd29a95f27338c1c8b026185472eef

[GitHub] [hudi] hudi-bot commented on pull request #4083: [HUDI-2837] Add support for using database name in incremental query

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1011017031 ## CI report: * 00221c82e8b1693280fd72625eafcd503d54323c UNKNOWN * 46053bb143d1fd1274ac466197cc9361708e738b UNKNOWN * 2722bfcfd29a95f27338c1c8b026185472eefba0 UNKN

[jira] [Updated] (HUDI-3234) Fixing read of an empty table (with first commit failed) should return empty RDD

2022-01-12 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3234: -- Description: When first commit fails, read of this table should return empty RDD. Pleas

[jira] [Updated] (HUDI-3234) Fixing read of an empty table (with first commit failed) should return empty RDD

2022-01-12 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3234: -- Sprint: Cont' improve - 2021/01/10 > Fixing read of an empty table (with first commit f

[GitHub] [hudi] dongkelun commented on pull request #4083: [HUDI-2837] Add support for using database name in incremental query

2022-01-12 Thread GitBox
dongkelun commented on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1011022143 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot commented on pull request #4083: [HUDI-2837] Add support for using database name in incremental query

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1011024426 ## CI report: * 00221c82e8b1693280fd72625eafcd503d54323c UNKNOWN * 46053bb143d1fd1274ac466197cc9361708e738b UNKNOWN * 2722bfcfd29a95f27338c1c8b026185472eefba0 UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4083: [HUDI-2837] Add support for using database name in incremental query

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1011017031 ## CI report: * 00221c82e8b1693280fd72625eafcd503d54323c UNKNOWN * 46053bb143d1fd1274ac466197cc9361708e738b UNKNOWN * 2722bfcfd29a95f27338c1c8b026185472eef

[jira] [Resolved] (HUDI-2944) Resolve ClosedChannelException due to Bit cask disk map

2022-01-12 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan resolved HUDI-2944. --- > Resolve ClosedChannelException due to Bit cask disk map > -

[GitHub] [hudi] hudi-bot removed a comment on pull request #4523: [WIP][HUDI-3173] Add INDEX action type and corresponding commit metadata

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4523: URL: https://github.com/apache/hudi/pull/4523#issuecomment-1006610017 ## CI report: * 700a87f4f67a1cac8f5b870882ab7b61628b4020 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4523: [WIP][HUDI-3173] Add INDEX action type and corresponding commit metadata

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4523: URL: https://github.com/apache/hudi/pull/4523#issuecomment-1011027765 ## CI report: * 700a87f4f67a1cac8f5b870882ab7b61628b4020 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[jira] [Commented] (HUDI-2874) hudi should remove the temp file which create by HoodieMergedLogRecordScanner， when we use hive/presto

2022-01-12 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17474511#comment-17474511 ] sivabalan narayanan commented on HUDI-2874: --- [~xiaotaotao] : if I am not wrong,

[jira] [Closed] (HUDI-2869) Metadata bootstrapping should ignore data files from partial/inflight commits

2022-01-12 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-2869. - Resolution: Duplicate > Metadata bootstrapping should ignore data files from partial/infli

[GitHub] [hudi] hudi-bot commented on pull request #4523: [WIP][HUDI-3173] Add INDEX action type and corresponding commit metadata

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4523: URL: https://github.com/apache/hudi/pull/4523#issuecomment-1011030599 ## CI report: * 700a87f4f67a1cac8f5b870882ab7b61628b4020 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4523: [WIP][HUDI-3173] Add INDEX action type and corresponding commit metadata

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4523: URL: https://github.com/apache/hudi/pull/4523#issuecomment-1011027765 ## CI report: * 700a87f4f67a1cac8f5b870882ab7b61628b4020 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[jira] [Updated] (HUDI-3072) AutoCommit misses to detect write conflicts during concurrent transactions

2022-01-12 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3072: -- Story Points: 1 (was: 8) > AutoCommit misses to detect write conflicts during concurren

[jira] [Assigned] (HUDI-3072) AutoCommit misses to detect write conflicts during concurrent transactions

2022-01-12 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-3072: - Assignee: sivabalan narayanan (was: Manoj Govindassamy) > AutoCommit misses to d

[jira] [Assigned] (HUDI-3234) Fixing read of an empty table (with first commit failed) should return empty RDD

2022-01-12 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-3234: - Assignee: sivabalan narayanan > Fixing read of an empty table (with first commit

[jira] [Updated] (HUDI-3072) AutoCommit misses to detect write conflicts during concurrent transactions

2022-01-12 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3072: -- Sprint: Cont' improve - 2021/01/10 > AutoCommit misses to detect write conflicts during

[GitHub] [hudi] codope opened a new pull request #4572: [HUDI-2943] Complete pending clustering before deltastreamer sync

2022-01-12 Thread GitBox
codope opened a new pull request #4572: URL: https://github.com/apache/hudi/pull/4572 ## What is the purpose of the pull request Fixes HUDI-2943 - Add config to retry last pending clustering before writing to sink - If the config set to true and inline clustering enabled, then c

[jira] [Updated] (HUDI-2943) Deltastreamer fails to continue with pending clustering after restart in 0.10.0 and inline clustering

2022-01-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2943: - Labels: core-flow-ds pull-request-available sev:high (was: core-flow-ds sev:high) > Deltastreame

[GitHub] [hudi] hudi-bot commented on pull request #4572: [HUDI-2943] Complete pending clustering before deltastreamer sync

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4572: URL: https://github.com/apache/hudi/pull/4572#issuecomment-1011045720 ## CI report: * 86716004ac41c76ed50e8728cbf06e068c9f7188 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot removed a comment on pull request #4572: [HUDI-2943] Complete pending clustering before deltastreamer sync

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4572: URL: https://github.com/apache/hudi/pull/4572#issuecomment-1011045720 ## CI report: * 86716004ac41c76ed50e8728cbf06e068c9f7188 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4572: [HUDI-2943] Complete pending clustering before deltastreamer sync

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4572: URL: https://github.com/apache/hudi/pull/4572#issuecomment-1011048545 ## CI report: * 86716004ac41c76ed50e8728cbf06e068c9f7188 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[jira] [Commented] (HUDI-2874) hudi should remove the temp file which create by HoodieMergedLogRecordScanner， when we use hive/presto

2022-01-12 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17474530#comment-17474530 ] Tao Meng commented on HUDI-2874: [~shivnarayan]   yes we have areadly fixed this issue in

[GitHub] [hudi] hudi-bot commented on pull request #4083: [HUDI-2837] Add support for using database name in incremental query

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1011075309 ## CI report: * 00221c82e8b1693280fd72625eafcd503d54323c UNKNOWN * 46053bb143d1fd1274ac466197cc9361708e738b UNKNOWN * 2722bfcfd29a95f27338c1c8b026185472eefba0 UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4083: [HUDI-2837] Add support for using database name in incremental query

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1011024426 ## CI report: * 00221c82e8b1693280fd72625eafcd503d54323c UNKNOWN * 46053bb143d1fd1274ac466197cc9361708e738b UNKNOWN * 2722bfcfd29a95f27338c1c8b026185472eef

[GitHub] [hudi] xiarixiaoyao commented on pull request #4540: [HUDI-3194] fix MOR snapshot query (HIVE) during compaction

2022-01-12 Thread GitBox
xiarixiaoyao commented on pull request #4540: URL: https://github.com/apache/hudi/pull/4540#issuecomment-1011081051 @codope could you pls review this pr again, thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [hudi] hudi-bot commented on pull request #4523: [WIP][HUDI-3173] Add INDEX action type and corresponding commit metadata

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4523: URL: https://github.com/apache/hudi/pull/4523#issuecomment-1011089899 ## CI report: * b2d14ae38d1da89caa89129d28d2d3032aa13610 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4523: [WIP][HUDI-3173] Add INDEX action type and corresponding commit metadata

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4523: URL: https://github.com/apache/hudi/pull/4523#issuecomment-1011030599 ## CI report: * 700a87f4f67a1cac8f5b870882ab7b61628b4020 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4572: [HUDI-2943] Complete pending clustering before deltastreamer sync

2022-01-12 Thread GitBox
hudi-bot commented on pull request #4572: URL: https://github.com/apache/hudi/pull/4572#issuecomment-104865 ## CI report: * 86716004ac41c76ed50e8728cbf06e068c9f7188 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4572: [HUDI-2943] Complete pending clustering before deltastreamer sync

2022-01-12 Thread GitBox
hudi-bot removed a comment on pull request #4572: URL: https://github.com/apache/hudi/pull/4572#issuecomment-1011048545 ## CI report: * 86716004ac41c76ed50e8728cbf06e068c9f7188 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] nsivabalan commented on a change in pull request #4572: [HUDI-2943] Complete pending clustering before deltastreamer sync

2022-01-12 Thread GitBox
nsivabalan commented on a change in pull request #4572: URL: https://github.com/apache/hudi/pull/4572#discussion_r783141256 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieDeltaStreamer.java ## @@ -369,6 +369,9 @@ private boolean onDelt

  1   2   3   4   >