[GitHub] [hudi] hudi-bot removed a comment on pull request #3391: [HUDI-83] Fix Timestamp type read by Hive

2022-01-28 Thread GitBox
hudi-bot removed a comment on pull request #3391: URL: https://github.com/apache/hudi/pull/3391#issuecomment-1023954251 ## CI report: * ecb72b89015831cfbfa99ebcb027f660729b3195 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #3391: [HUDI-83] Fix Timestamp type read by Hive

2022-01-28 Thread GitBox
hudi-bot commented on pull request #3391: URL: https://github.com/apache/hudi/pull/3391#issuecomment-1023995465 ## CI report: * 223c320447bc9adc8fccaabb9c590bed159b375d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[jira] [Comment Edited] (HUDI-3335) Loading Hudi table fails with NullPointerException

2022-01-28 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17483603#comment-17483603 ] Harsha Teja Kanna edited comment on HUDI-3335 at 1/28/22, 8:34 AM: -

[GitHub] [hudi] hudi-bot removed a comment on pull request #4709: [HUDI-3338] custom relation instead of HadoopFsRelation

2022-01-28 Thread GitBox
hudi-bot removed a comment on pull request #4709: URL: https://github.com/apache/hudi/pull/4709#issuecomment-1023961928 ## CI report: * 2f14cbdd761921dc1b29c01b1201f58cc1f98b5a UNKNOWN * 8f670f3466a15e536605b67edd5586c152d04035 Azure: [CANCELED](https://dev.azure.com/apache-hu

[GitHub] [hudi] hudi-bot commented on pull request #4709: [HUDI-3338] custom relation instead of HadoopFsRelation

2022-01-28 Thread GitBox
hudi-bot commented on pull request #4709: URL: https://github.com/apache/hudi/pull/4709#issuecomment-1024005458 ## CI report: * 2f14cbdd761921dc1b29c01b1201f58cc1f98b5a UNKNOWN * 879e966586fe287e710fb2b9db7a2436fef03a92 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] YannByron commented on pull request #4709: [HUDI-3338] custom relation instead of HadoopFsRelation

2022-01-28 Thread GitBox
YannByron commented on pull request #4709: URL: https://github.com/apache/hudi/pull/4709#issuecomment-1024024406 @nsivabalan @leesf @xushiyan can you have a chance to review this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [hudi] YannByron commented on pull request #4608: [HUDI-3253] preferred to use the table's own location

2022-01-28 Thread GitBox
YannByron commented on pull request #4608: URL: https://github.com/apache/hudi/pull/4608#issuecomment-1024025320 @leesf @xushiyan @XuQianJin-Stars can you have a chance to review this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] manojpec opened a new pull request #4711: [WIP][HUDI-1295][HUDI-3166] Hoodie Index Type Metadata Bloom implementation

2022-01-28 Thread GitBox
manojpec opened a new pull request #4711: URL: https://github.com/apache/hudi/pull/4711 ## What is the purpose of the pull request New Hoodie Index Type - Metadata Bloom implementation. This is based out of the bloom filters and column stats index maintained under the metadata table.

[GitHub] [hudi] hudi-bot commented on pull request #4711: [WIP][HUDI-1295][HUDI-3166] Hoodie Index Type Metadata Bloom implementation

2022-01-28 Thread GitBox
hudi-bot commented on pull request #4711: URL: https://github.com/apache/hudi/pull/4711#issuecomment-1024053500 ## CI report: * 1f2e400c0f77ac70906cba487f31cc9b9daf7915 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot removed a comment on pull request #4711: [WIP][HUDI-1295][HUDI-3166] Hoodie Index Type Metadata Bloom implementation

2022-01-28 Thread GitBox
hudi-bot removed a comment on pull request #4711: URL: https://github.com/apache/hudi/pull/4711#issuecomment-1024053500 ## CI report: * 1f2e400c0f77ac70906cba487f31cc9b9daf7915 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4711: [WIP][HUDI-1295][HUDI-3166] Hoodie Index Type Metadata Bloom implementation

2022-01-28 Thread GitBox
hudi-bot commented on pull request #4711: URL: https://github.com/apache/hudi/pull/4711#issuecomment-1024055788 ## CI report: * 1f2e400c0f77ac70906cba487f31cc9b9daf7915 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] andykrk edited a comment on issue #4604: [SUPPORT] Archive functionality fails

2022-01-28 Thread GitBox
andykrk edited a comment on issue #4604: URL: https://github.com/apache/hudi/issues/4604#issuecomment-1024091806 @nsivabalan I was able to reproduce the issue on the other table. Let me add some additional information to it that may help. We start with our current table that was created

[GitHub] [hudi] andykrk commented on issue #4604: [SUPPORT] Archive functionality fails

2022-01-28 Thread GitBox
andykrk commented on issue #4604: URL: https://github.com/apache/hudi/issues/4604#issuecomment-1024091806 @nsivabalan I was able to reproduce the issue on the other table. Let me add some additional information to it that may help. We start with our current table that was created with hu

[GitHub] [hudi] codope opened a new pull request #4712: [HUDI-2809] Introduce a checksum mechanism for validating hoodie.properties

2022-01-28 Thread GitBox
codope opened a new pull request #4712: URL: https://github.com/apache/hudi/pull/4712 ## What is the purpose of the pull request To detect partial writes on HDFS, this PR adds a new property which gets appended at the end of hoodie.properties file while creating or modiying table co

[jira] [Updated] (HUDI-2809) Introduce a checksum mechanism for validating hoodie.properties

2022-01-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2809: - Labels: pull-request-available (was: ) > Introduce a checksum mechanism for validating hoodie.pro

[GitHub] [hudi] hudi-bot removed a comment on pull request #4711: [WIP][HUDI-1295][HUDI-3166] Hoodie Index Type Metadata Bloom implementation

2022-01-28 Thread GitBox
hudi-bot removed a comment on pull request #4711: URL: https://github.com/apache/hudi/pull/4711#issuecomment-1024055788 ## CI report: * 1f2e400c0f77ac70906cba487f31cc9b9daf7915 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4711: [WIP][HUDI-1295][HUDI-3166] Hoodie Index Type Metadata Bloom implementation

2022-01-28 Thread GitBox
hudi-bot commented on pull request #4711: URL: https://github.com/apache/hudi/pull/4711#issuecomment-1024104513 ## CI report: * 1f2e400c0f77ac70906cba487f31cc9b9daf7915 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #4712: [HUDI-2809] Introduce a checksum mechanism for validating hoodie.properties

2022-01-28 Thread GitBox
hudi-bot commented on pull request #4712: URL: https://github.com/apache/hudi/pull/4712#issuecomment-1024104556 ## CI report: * 233e267344d8094313bb7e24e65cd7db2e3c0672 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot removed a comment on pull request #4712: [HUDI-2809] Introduce a checksum mechanism for validating hoodie.properties

2022-01-28 Thread GitBox
hudi-bot removed a comment on pull request #4712: URL: https://github.com/apache/hudi/pull/4712#issuecomment-1024104556 ## CI report: * 233e267344d8094313bb7e24e65cd7db2e3c0672 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4712: [HUDI-2809] Introduce a checksum mechanism for validating hoodie.properties

2022-01-28 Thread GitBox
hudi-bot commented on pull request #4712: URL: https://github.com/apache/hudi/pull/4712#issuecomment-1024106764 ## CI report: * 233e267344d8094313bb7e24e65cd7db2e3c0672 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] codope closed pull request #4484: [HUDI-3097][WIP] Allow hbase-shaded-server in trino bundle

2022-01-28 Thread GitBox
codope closed pull request #4484: URL: https://github.com/apache/hudi/pull/4484 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr..

[GitHub] [hudi] codope commented on a change in pull request #3957: [HUDI-2688][RFC-40] A new Hudi connector for Trino

2022-01-28 Thread GitBox
codope commented on a change in pull request #3957: URL: https://github.com/apache/hudi/pull/3957#discussion_r794426692 ## File path: rfc/rfc-40/rfc-40.md ## @@ -0,0 +1,195 @@ + + +# RFC-40: Hudi Connector for Trino + +## Proposers + +- @codope + +## Approvers + +- @bvaradar +-

[GitHub] [hudi] codope commented on a change in pull request #3957: [HUDI-2688][RFC-40] A new Hudi connector for Trino

2022-01-28 Thread GitBox
codope commented on a change in pull request #3957: URL: https://github.com/apache/hudi/pull/3957#discussion_r794432109 ## File path: rfc/rfc-40/rfc-40.md ## @@ -0,0 +1,195 @@ + + +# RFC-40: Hudi Connector for Trino + +## Proposers + +- @codope + +## Approvers + +- @bvaradar +-

[jira] [Created] (HUDI-3339) Reuse or implement caching like the hive connector

2022-01-28 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-3339: - Summary: Reuse or implement caching like the hive connector Key: HUDI-3339 URL: https://issues.apache.org/jira/browse/HUDI-3339 Project: Apache Hudi Issue Type: Ta

[GitHub] [hudi] codope commented on a change in pull request #3957: [HUDI-2688][RFC-40] A new Hudi connector for Trino

2022-01-28 Thread GitBox
codope commented on a change in pull request #3957: URL: https://github.com/apache/hudi/pull/3957#discussion_r794432686 ## File path: rfc/rfc-40/rfc-40.md ## @@ -0,0 +1,195 @@ + + +# RFC-40: Hudi Connector for Trino + +## Proposers + +- @codope + +## Approvers + +- @bvaradar +-

[GitHub] [hudi] rafcis02 commented on issue #4552: [BUG] Data corrupted in the timestamp field to 1970-01-01 19:45:30.000 after subsequent upsert run

2022-01-28 Thread GitBox
rafcis02 commented on issue #4552: URL: https://github.com/apache/hudi/issues/4552#issuecomment-1024142956 I've tried it for BULK_INSERT and UPSERT as well, bot nothing works for me. I prepared sample test job of that so you can reproduce it or just review it (I hope I just misconfig

[GitHub] [hudi] hudi-bot commented on pull request #3957: [HUDI-2688][RFC-40] A new Hudi connector for Trino

2022-01-28 Thread GitBox
hudi-bot commented on pull request #3957: URL: https://github.com/apache/hudi/pull/3957#issuecomment-1024155643 ## CI report: * 9b2b86aade67c389e9835455cbd77a2b8ea5c92b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #3957: [HUDI-2688][RFC-40] A new Hudi connector for Trino

2022-01-28 Thread GitBox
hudi-bot removed a comment on pull request #3957: URL: https://github.com/apache/hudi/pull/3957#issuecomment-1021753102 ## CI report: * 9b2b86aade67c389e9835455cbd77a2b8ea5c92b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #3957: [HUDI-2688][RFC-40] A new Hudi connector for Trino

2022-01-28 Thread GitBox
hudi-bot commented on pull request #3957: URL: https://github.com/apache/hudi/pull/3957#issuecomment-1024158147 ## CI report: * 9b2b86aade67c389e9835455cbd77a2b8ea5c92b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #3957: [HUDI-2688][RFC-40] A new Hudi connector for Trino

2022-01-28 Thread GitBox
hudi-bot removed a comment on pull request #3957: URL: https://github.com/apache/hudi/pull/3957#issuecomment-1024155643 ## CI report: * 9b2b86aade67c389e9835455cbd77a2b8ea5c92b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] codope commented on pull request #4523: [WIP][HUDI-3173] Add INDEX action type and corresponding commit metadata

2022-01-28 Thread GitBox
codope commented on pull request #4523: URL: https://github.com/apache/hudi/pull/4523#issuecomment-1024159578 Closing it in favor of #4693 which has the core functionality and all comments from this PR addressed. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] codope closed pull request #4523: [WIP][HUDI-3173] Add INDEX action type and corresponding commit metadata

2022-01-28 Thread GitBox
codope closed pull request #4523: URL: https://github.com/apache/hudi/pull/4523 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr..

[GitHub] [hudi] stym06 opened a new issue #4713: [SUPPORT] Executor OOM upserting 20M records from Kafka

2022-01-28 Thread GitBox
stym06 opened a new issue #4713: URL: https://github.com/apache/hudi/issues/4713 **Describe the problem you faced** While upserting Mongo oplogs from Kafka to Blob, facing Executor OOM **Environment Description** * Hudi version : 0.9.0 * Spark version : 2.4.4

[GitHub] [hudi] hudi-bot removed a comment on pull request #4699: [HUDI-3336][HUDI-FLINK] Configurations transferred through Flink SQL canno…

2022-01-28 Thread GitBox
hudi-bot removed a comment on pull request #4699: URL: https://github.com/apache/hudi/pull/4699#issuecomment-1023369398 ## CI report: * 8da29b1d0e42ea7879d2624ab6a9e008a9892c99 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4699: [HUDI-3336][HUDI-FLINK] Configurations transferred through Flink SQL canno…

2022-01-28 Thread GitBox
hudi-bot commented on pull request #4699: URL: https://github.com/apache/hudi/pull/4699#issuecomment-1024168298 ## CI report: * 8da29b1d0e42ea7879d2624ab6a9e008a9892c99 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[jira] [Created] (HUDI-3340) Fix deploy_staging_jars for diff spark versions

2022-01-28 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-3340: - Summary: Fix deploy_staging_jars for diff spark versions Key: HUDI-3340 URL: https://issues.apache.org/jira/browse/HUDI-3340 Project: Apache Hudi I

[jira] [Updated] (HUDI-3336) Configurations transferred through Flink SQL cannot take effect

2022-01-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3336: - Labels: pull-request-available (was: ) > Configurations transferred through Flink SQL cannot take

[jira] [Updated] (HUDI-3340) Fix deploy_staging_jars for diff spark versions

2022-01-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3340: -- Priority: Critical (was: Major) > Fix deploy_staging_jars for diff spark versions > ---

[jira] [Assigned] (HUDI-3340) Fix deploy_staging_jars for diff spark versions

2022-01-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-3340: - Assignee: sivabalan narayanan > Fix deploy_staging_jars for diff spark versions >

[GitHub] [hudi] hudi-bot removed a comment on pull request #4699: [HUDI-3336][HUDI-FLINK] Configurations transferred through Flink SQL canno…

2022-01-28 Thread GitBox
hudi-bot removed a comment on pull request #4699: URL: https://github.com/apache/hudi/pull/4699#issuecomment-1024168298 ## CI report: * 8da29b1d0e42ea7879d2624ab6a9e008a9892c99 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4699: [HUDI-3336][HUDI-FLINK] Configurations transferred through Flink SQL canno…

2022-01-28 Thread GitBox
hudi-bot commented on pull request #4699: URL: https://github.com/apache/hudi/pull/4699#issuecomment-1024170791 ## CI report: * 8da29b1d0e42ea7879d2624ab6a9e008a9892c99 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4712: [HUDI-2809] Introduce a checksum mechanism for validating hoodie.properties

2022-01-28 Thread GitBox
hudi-bot removed a comment on pull request #4712: URL: https://github.com/apache/hudi/pull/4712#issuecomment-1024106764 ## CI report: * 233e267344d8094313bb7e24e65cd7db2e3c0672 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4712: [HUDI-2809] Introduce a checksum mechanism for validating hoodie.properties

2022-01-28 Thread GitBox
hudi-bot commented on pull request #4712: URL: https://github.com/apache/hudi/pull/4712#issuecomment-1024170850 ## CI report: * 233e267344d8094313bb7e24e65cd7db2e3c0672 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4572: [HUDI-2943] Complete pending clustering before deltastreamer sync

2022-01-28 Thread GitBox
hudi-bot removed a comment on pull request #4572: URL: https://github.com/apache/hudi/pull/4572#issuecomment-1012189079 ## CI report: * 1e09793a27df6e239065aadee35ec006e5e9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4572: [HUDI-2943] Complete pending clustering before deltastreamer sync

2022-01-28 Thread GitBox
hudi-bot commented on pull request #4572: URL: https://github.com/apache/hudi/pull/4572#issuecomment-1024225358 ## CI report: * 1e09793a27df6e239065aadee35ec006e5e9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4572: [HUDI-2943] Complete pending clustering before deltastreamer sync

2022-01-28 Thread GitBox
hudi-bot removed a comment on pull request #4572: URL: https://github.com/apache/hudi/pull/4572#issuecomment-1024225358 ## CI report: * 1e09793a27df6e239065aadee35ec006e5e9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4572: [HUDI-2943] Complete pending clustering before deltastreamer sync

2022-01-28 Thread GitBox
hudi-bot commented on pull request #4572: URL: https://github.com/apache/hudi/pull/4572#issuecomment-1024228027 ## CI report: * 1e09793a27df6e239065aadee35ec006e5e9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #3957: [HUDI-2688][RFC-40] A new Hudi connector for Trino

2022-01-28 Thread GitBox
hudi-bot removed a comment on pull request #3957: URL: https://github.com/apache/hudi/pull/3957#issuecomment-1024158147 ## CI report: * 9b2b86aade67c389e9835455cbd77a2b8ea5c92b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #3957: [HUDI-2688][RFC-40] A new Hudi connector for Trino

2022-01-28 Thread GitBox
hudi-bot commented on pull request #3957: URL: https://github.com/apache/hudi/pull/3957#issuecomment-1024236938 ## CI report: * 341cc4daa53d2f97ab04a44d954652e79569e406 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] codope merged pull request #3957: [HUDI-2688][RFC-40] A new Hudi connector for Trino

2022-01-28 Thread GitBox
codope merged pull request #3957: URL: https://github.com/apache/hudi/pull/3957 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr..

[hudi] branch master updated (0bd38f2 -> 2b52a56)

2022-01-28 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 0bd38f2 [HUDI-2596] Make class names consistent in hudi-client (#4680) add 2b52a56 [HUDI-2688][RFC-40] A new Hud

[GitHub] [hudi] hudi-bot removed a comment on pull request #4699: [HUDI-3336][HUDI-FLINK] Configurations transferred through Flink SQL canno…

2022-01-28 Thread GitBox
hudi-bot removed a comment on pull request #4699: URL: https://github.com/apache/hudi/pull/4699#issuecomment-1024170791 ## CI report: * 8da29b1d0e42ea7879d2624ab6a9e008a9892c99 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4699: [HUDI-3336][HUDI-FLINK] Configurations transferred through Flink SQL canno…

2022-01-28 Thread GitBox
hudi-bot commented on pull request #4699: URL: https://github.com/apache/hudi/pull/4699#issuecomment-1024249663 ## CI report: * 6e9036e89041d7ee6cf995b29502cbc10bfbff8d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] codope commented on a change in pull request #2982: [HUDI-1441] Fixing HoodieAvroUtils.rewriteRecord for nested record schema evolution

2022-01-28 Thread GitBox
codope commented on a change in pull request #2982: URL: https://github.com/apache/hudi/pull/2982#discussion_r794524309 ## File path: hudi-common/src/test/java/org/apache/hudi/avro/TestHoodieAvroUtils.java ## @@ -236,4 +237,81 @@ public void testGetNestedFieldVal() { }

[GitHub] [hudi] hudi-bot removed a comment on pull request #4572: [HUDI-2943] Complete pending clustering before deltastreamer sync

2022-01-28 Thread GitBox
hudi-bot removed a comment on pull request #4572: URL: https://github.com/apache/hudi/pull/4572#issuecomment-1024228027 ## CI report: * 1e09793a27df6e239065aadee35ec006e5e9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4572: [HUDI-2943] Complete pending clustering before deltastreamer sync

2022-01-28 Thread GitBox
hudi-bot commented on pull request #4572: URL: https://github.com/apache/hudi/pull/4572#issuecomment-1024285991 ## CI report: * 38433f8184b92c7134e0dbd0fdffa8d399bd93a6 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] harishraju-govindaraju closed issue #4641: [SUPPORT] - HudiDeltaStreamer - EMR - SparkSubmit Not working

2022-01-28 Thread GitBox
harishraju-govindaraju closed issue #4641: URL: https://github.com/apache/hudi/issues/4641 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits

[GitHub] [hudi] harishraju-govindaraju commented on issue #4641: [SUPPORT] - HudiDeltaStreamer - EMR - SparkSubmit Not working

2022-01-28 Thread GitBox
harishraju-govindaraju commented on issue #4641: URL: https://github.com/apache/hudi/issues/4641#issuecomment-1024294748 I was able to do spark submit. My parameters had something wrong. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hudi] YannByron opened a new pull request #4714: [HUDI-3204] fix problem that spark on TimestampKeyGenerator has no re…

2022-01-28 Thread GitBox
YannByron opened a new pull request #4714: URL: https://github.com/apache/hudi/pull/4714 …sult when query by partition column ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening

[jira] [Updated] (HUDI-3204) spark on TimestampBasedKeyGenerator has no result when query by partition column

2022-01-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3204: - Labels: hudi-on-call pull-request-available sev:critical (was: hudi-on-call sev:critical) > spar

[GitHub] [hudi] hudi-bot commented on pull request #4714: [HUDI-3204] fix problem that spark on TimestampKeyGenerator has no re…

2022-01-28 Thread GitBox
hudi-bot commented on pull request #4714: URL: https://github.com/apache/hudi/pull/4714#issuecomment-1024319777 ## CI report: * 1dc6c2f464e45fcbffe5d7fde2fbb1c66a6fca34 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot commented on pull request #4714: [HUDI-3204] fix problem that spark on TimestampKeyGenerator has no re…

2022-01-28 Thread GitBox
hudi-bot commented on pull request #4714: URL: https://github.com/apache/hudi/pull/4714#issuecomment-1024322851 ## CI report: * 1dc6c2f464e45fcbffe5d7fde2fbb1c66a6fca34 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4714: [HUDI-3204] fix problem that spark on TimestampKeyGenerator has no re…

2022-01-28 Thread GitBox
hudi-bot removed a comment on pull request #4714: URL: https://github.com/apache/hudi/pull/4714#issuecomment-1024319777 ## CI report: * 1dc6c2f464e45fcbffe5d7fde2fbb1c66a6fca34 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] xushiyan commented on a change in pull request #3893: [HUDI-2656] Generalize HoodieIndex for flexible record data type

2022-01-28 Thread GitBox
xushiyan commented on a change in pull request #3893: URL: https://github.com/apache/hudi/pull/3893#discussion_r794618101 ## File path: hudi-common/src/main/java/org/apache/hudi/common/model/HoodieRecord.java ## @@ -18,21 +18,21 @@ package org.apache.hudi.common.model; -i

[GitHub] [hudi] hudi-bot removed a comment on pull request #4704: [HUDI-3330] Remove fixture test tables for multi writer tests

2022-01-28 Thread GitBox
hudi-bot removed a comment on pull request #4704: URL: https://github.com/apache/hudi/pull/4704#issuecomment-1023906918 ## CI report: * c13c56e14dad9fad992fdf4a50e24e45c1539817 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4704: [HUDI-3330] Remove fixture test tables for multi writer tests

2022-01-28 Thread GitBox
hudi-bot commented on pull request #4704: URL: https://github.com/apache/hudi/pull/4704#issuecomment-1024345858 ## CI report: * c13c56e14dad9fad992fdf4a50e24e45c1539817 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot removed a comment on pull request #4704: [HUDI-3330] Remove fixture test tables for multi writer tests

2022-01-28 Thread GitBox
hudi-bot removed a comment on pull request #4704: URL: https://github.com/apache/hudi/pull/4704#issuecomment-1024345858 ## CI report: * c13c56e14dad9fad992fdf4a50e24e45c1539817 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4704: [HUDI-3330] Remove fixture test tables for multi writer tests

2022-01-28 Thread GitBox
hudi-bot commented on pull request #4704: URL: https://github.com/apache/hudi/pull/4704#issuecomment-1024348844 ## CI report: * c13c56e14dad9fad992fdf4a50e24e45c1539817 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] xushiyan commented on a change in pull request #4697: [HUDI-3318] Drafted RFC-46

2022-01-28 Thread GitBox
xushiyan commented on a change in pull request #4697: URL: https://github.com/apache/hudi/pull/4697#discussion_r794632615 ## File path: rfc/rfc-46/rfc-46.md ## @@ -0,0 +1,159 @@ + +# RFC-46: Optimize Record Payload handling + +## Proposers + +- @alexeykudinkin + +## Approvers +

[GitHub] [hudi] yihua merged pull request #4707: [Docs] Stop-gap solution to fix the broken blog link

2022-01-28 Thread GitBox
yihua merged pull request #4707: URL: https://github.com/apache/hudi/pull/4707 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[hudi] branch asf-site updated: [Docs] Stop-gap solution to fix the broken blog link (#4707)

2022-01-28 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 9b8ff0e [Docs] Stop-gap solution to fix the br

[GitHub] [hudi] nsivabalan commented on issue #4641: [SUPPORT] - HudiDeltaStreamer - EMR - SparkSubmit Not working

2022-01-28 Thread GitBox
nsivabalan commented on issue #4641: URL: https://github.com/apache/hudi/issues/4641#issuecomment-1024372275 got it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[hudi] branch asf-site updated: [MINOR] Adjust Hudi CLI commands for compaction in Docker Demo (#4696)

2022-01-28 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new dbce9ef [MINOR] Adjust Hudi CLI commands for c

[GitHub] [hudi] yihua merged pull request #4696: [MINOR] Adjust Hudi CLI commands for compaction in Docker Demo

2022-01-28 Thread GitBox
yihua merged pull request #4696: URL: https://github.com/apache/hudi/pull/4696 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[GitHub] [hudi] yihua closed pull request #4706: [MINOR][DO NOT MERGE] Debug asf-site CI

2022-01-28 Thread GitBox
yihua closed pull request #4706: URL: https://github.com/apache/hudi/pull/4706 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[jira] [Commented] (HUDI-512) Support for Index functions on columns to generate logical or micro partitioning

2022-01-28 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17483836#comment-17483836 ] Sagar Sumit commented on HUDI-512: -- Let’s call *function(column) <= value* a {color:#FFAB0

[jira] [Created] (HUDI-3341) Investigate that metadata table cannot be read for hadoop-aws 2.7.x

2022-01-28 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-3341: --- Summary: Investigate that metadata table cannot be read for hadoop-aws 2.7.x Key: HUDI-3341 URL: https://issues.apache.org/jira/browse/HUDI-3341 Project: Apache Hudi

[jira] [Updated] (HUDI-2458) Relax compaction in metadata being fenced based on inflight requests in data table

2022-01-28 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-2458: Story Points: 2 > Relax compaction in metadata being fenced based on inflight requests in data > table > --

[jira] [Updated] (HUDI-3174) Implement metadata filesystem view changes to support INDEX action type

2022-01-28 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3174: -- Status: Patch Available (was: In Progress) > Implement metadata filesystem view changes to support INDE

[jira] [Updated] (HUDI-3341) Investigate that metadata table cannot be read for hadoop-aws 2.7.x

2022-01-28 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3341: Sprint: Hudi-Sprint-Jan-24 > Investigate that metadata table cannot be read for hadoop-aws 2.7.x > -

[jira] [Updated] (HUDI-2930) Rollbacks are not archived when metadata table is enabled

2022-01-28 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-2930: Story Points: 2 > Rollbacks are not archived when metadata table is enabled > --

[jira] [Updated] (HUDI-2809) Introduce a checksum mechanism for validating hoodie.properties

2022-01-28 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-2809: -- Status: Patch Available (was: In Progress) > Introduce a checksum mechanism for validating hoodie.prope

[jira] [Updated] (HUDI-3207) Hudi Trino connector PR review

2022-01-28 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3207: -- Status: Patch Available (was: In Progress) > Hudi Trino connector PR review > -

[jira] [Updated] (HUDI-2961) Async table services can race with metadata table updates

2022-01-28 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-2961: Story Points: 2 > Async table services can race with metadata table updates > --

[jira] [Updated] (HUDI-3225) RFC for Async Metadata Index

2022-01-28 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3225: -- Status: Patch Available (was: In Progress) > RFC for Async Metadata Index > ---

[jira] [Updated] (HUDI-2708) Support indexing of metadata table even when async table service is in progress

2022-01-28 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-2708: -- Status: In Progress (was: Open) > Support indexing of metadata table even when async table service is i

[jira] [Updated] (HUDI-3341) Investigate that metadata table cannot be read for hadoop-aws 2.7.x

2022-01-28 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3341: Status: In Progress (was: Open) > Investigate that metadata table cannot be read for hadoop-aws 2.7.x > ---

[jira] [Updated] (HUDI-3275) Add tests for async metadata indexing

2022-01-28 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3275: -- Status: In Progress (was: Open) > Add tests for async metadata indexing > -

[jira] [Updated] (HUDI-1370) Scoping work needed to support bootstrapped data table and RFC-15 together

2022-01-28 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-1370: Status: In Progress (was: Open) > Scoping work needed to support bootstrapped data table and RFC-15 togethe

[jira] [Updated] (HUDI-3337) ParquetUtils fails extracting Parquet Column Range Metadata

2022-01-28 Thread Rajesh Mahindra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Mahindra updated HUDI-3337: -- Status: In Progress (was: Open) > ParquetUtils fails extracting Parquet Column Range Metadata >

[jira] [Updated] (HUDI-3322) Rollback Plan for Delta Commits constructed incorrectly

2022-01-28 Thread Rajesh Mahindra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Mahindra updated HUDI-3322: -- Status: In Progress (was: Open) > Rollback Plan for Delta Commits constructed incorrectly > ---

[jira] [Updated] (HUDI-2708) Support indexing of metadata table even when async table service is in progress

2022-01-28 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-2708: -- Status: Patch Available (was: In Progress) > Support indexing of metadata table even when async table s

[jira] [Updated] (HUDI-2708) Support indexing of metadata table even when async table service is in progress

2022-01-28 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-2708: -- Epic Link: HUDI-2488 (was: HUDI-1292) > Support indexing of metadata table even when async table servic

[jira] [Updated] (HUDI-2809) Introduce a checksum mechanism for validating hoodie.properties

2022-01-28 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-2809: -- Reviewers: sivabalan narayanan > Introduce a checksum mechanism for validating hoodie.properties > -

[jira] [Updated] (HUDI-2708) Support indexing of metadata table even when async table service is in progress

2022-01-28 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-2708: -- Reviewers: Manoj Govindassamy > Support indexing of metadata table even when async table service is in

[jira] [Updated] (HUDI-3316) HoodieColumnRangeMetadata doesn't include all statistics for the column

2022-01-28 Thread Rajesh Mahindra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Mahindra updated HUDI-3316: -- Status: Patch Available (was: In Progress) > HoodieColumnRangeMetadata doesn't include all stat

[jira] [Updated] (HUDI-3166) Implement new HoodieIndex based on metadata indices

2022-01-28 Thread Rajesh Mahindra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Mahindra updated HUDI-3166: -- Status: Patch Available (was: In Progress) > Implement new HoodieIndex based on metadata indice

[jira] [Updated] (HUDI-3175) Support INDEX action in write client

2022-01-28 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3175: -- Reviewers: Manoj Govindassamy > Support INDEX action in write client > -

[jira] [Resolved] (HUDI-2714) Benchmark MetaIndex performance w/ bloom and column stat metadata

2022-01-28 Thread Rajesh Mahindra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Mahindra resolved HUDI-2714. --- > Benchmark MetaIndex performance w/ bloom and column stat metadata > ---

[jira] [Closed] (HUDI-2763) Metadata table records key deduplication

2022-01-28 Thread Rajesh Mahindra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Mahindra closed HUDI-2763. - Resolution: Fixed > Metadata table records key deduplication > ---

  1   2   3   >