[GitHub] [hudi] manojpec commented on a change in pull request #4067: [HUDI-2763] Metadata table records key deduplication

2021-12-12 Thread GitBox
manojpec commented on a change in pull request #4067: URL: https://github.com/apache/hudi/pull/4067#discussion_r767238137 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieLogBlockFactory.java ## @@ -0,0 +1,90 @@ +/* + * Licensed to the Apach

[GitHub] [hudi] manojpec commented on a change in pull request #4067: [HUDI-2763] Metadata table records key deduplication

2021-12-12 Thread GitBox
manojpec commented on a change in pull request #4067: URL: https://github.com/apache/hudi/pull/4067#discussion_r767238296 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieMergedLogRecordScanner.java ## @@ -282,12 +286,35 @@ public Builder withPart

[GitHub] [hudi] leesf commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

2021-12-12 Thread GitBox
leesf commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767238312 ## File path: hudi-spark-datasource/hudi-spark3/src/main/java/org/apache/hudi/spark3/internal/ReflectUtil.java ## @@ -43,4 +47,23 @@ public static InsertIntoS

[GitHub] [hudi] manojpec commented on a change in pull request #4067: [HUDI-2763] Metadata table records key deduplication

2021-12-12 Thread GitBox
manojpec commented on a change in pull request #4067: URL: https://github.com/apache/hudi/pull/4067#discussion_r767238330 ## File path: hudi-common/src/main/java/org/apache/hudi/common/config/HoodieMetadataConfig.java ## @@ -137,6 +136,12 @@ .withDocumentation("There ar

[GitHub] [hudi] manojpec commented on a change in pull request #4067: [HUDI-2763] Metadata table records key deduplication

2021-12-12 Thread GitBox
manojpec commented on a change in pull request #4067: URL: https://github.com/apache/hudi/pull/4067#discussion_r767238351 ## File path: hudi-common/src/main/java/org/apache/hudi/common/config/HoodieMetadataConfig.java ## @@ -137,6 +136,12 @@ .withDocumentation("There ar

[GitHub] [hudi] manojpec commented on a change in pull request #4067: [HUDI-2763] Metadata table records key deduplication

2021-12-12 Thread GitBox
manojpec commented on a change in pull request #4067: URL: https://github.com/apache/hudi/pull/4067#discussion_r767238401 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java ## @@ -98,16 +101,13 @@ public HoodieLogFileReader(FileSy

[GitHub] [hudi] leesf commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

2021-12-12 Thread GitBox
leesf commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767238485 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/CreateHoodieTableAsSelectCommand.scala ## @@ -41,6 +41,10 @@ c

[GitHub] [hudi] leesf commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

2021-12-12 Thread GitBox
leesf commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767238556 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/HoodieLeafRunnableCommand.scala ## @@ -0,0 +1,24 @@ +/* + * Li

[GitHub] [hudi] nsivabalan commented on issue #3755: [Delta Streamer] file name mismatch with meta when compaction running

2021-12-12 Thread GitBox
nsivabalan commented on issue #3755: URL: https://github.com/apache/hudi/issues/3755#issuecomment-991852883 We made some fix around illegal arg exception you have posted above. Can you try with 0.10.0. Also, interested to understand more details around "after running few hours, the com

[GitHub] [hudi] nsivabalan commented on issue #3731: [SUPPORT] Concurrent write (OCC) on distinct partitions random errors

2021-12-12 Thread GitBox
nsivabalan commented on issue #3731: URL: https://github.com/apache/hudi/issues/3731#issuecomment-991853688 can you please try 0.10.0 and let us know how it goes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[GitHub] [hudi] hudi-bot removed a comment on pull request #4067: [HUDI-2763] Metadata table records key deduplication

2021-12-12 Thread GitBox
hudi-bot removed a comment on pull request #4067: URL: https://github.com/apache/hudi/pull/4067#issuecomment-991849212 ## CI report: * 3f3a24beec4a6b348e845ca374a74d487a9e1cf8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4067: [HUDI-2763] Metadata table records key deduplication

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4067: URL: https://github.com/apache/hudi/pull/4067#issuecomment-991854687 ## CI report: * 993240dc681e5fd011afbec6ce01ac2ee60fc8b8 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot commented on pull request #4270: [HUDI-2811] Support Spark 3.2

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-991857145 ## CI report: * e515cb1eadc04d0c5d04cf170e4af4724f5bd336 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4270: [HUDI-2811] Support Spark 3.2

2021-12-12 Thread GitBox
hudi-bot removed a comment on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-991850999 ## CI report: * e515cb1eadc04d0c5d04cf170e4af4724f5bd336 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] YannByron commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

2021-12-12 Thread GitBox
YannByron commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767251543 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/CreateHoodieTableAsSelectCommand.scala ## @@ -41,6 +41,10

[jira] [Created] (HUDI-2990) Delete partition without metadata sync to hms

2021-12-12 Thread Forward Xu (Jira)
Forward Xu created HUDI-2990: Summary: Delete partition without metadata sync to hms Key: HUDI-2990 URL: https://issues.apache.org/jira/browse/HUDI-2990 Project: Apache Hudi Issue Type: Bug

[jira] [Updated] (HUDI-2990) Delete partitions without metadata sync to hms

2021-12-12 Thread Forward Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Forward Xu updated HUDI-2990: - Summary: Delete partitions without metadata sync to hms (was: Delete partition without metadata sync to h

[GitHub] [hudi] YannByron commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

2021-12-12 Thread GitBox
YannByron commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767253317 ## File path: hudi-spark-datasource/hudi-spark3/src/main/java/org/apache/hudi/spark3/internal/ReflectUtil.java ## @@ -43,4 +47,23 @@ public static InsertI

[GitHub] [hudi] XuQianJin-Stars opened a new pull request #4291: Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
XuQianJin-Stars opened a new pull request #4291: URL: https://github.com/apache/hudi/pull/4291 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

[GitHub] [hudi] hudi-bot commented on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991873548 ## CI report: * fb90953569c20a3c2d57157c8729d4f38986ae13 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[jira] [Updated] (HUDI-2990) Delete partitions without metadata sync to hms

2021-12-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2990: - Labels: pull-request-available (was: ) > Delete partitions without metadata sync to hms > ---

[GitHub] [hudi] hudi-bot removed a comment on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
hudi-bot removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991873548 ## CI report: * fb90953569c20a3c2d57157c8729d4f38986ae13 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4270: [HUDI-2811] Support Spark 3.2

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-991873863 ## CI report: * e515cb1eadc04d0c5d04cf170e4af4724f5bd336 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot commented on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991873871 ## CI report: * fb90953569c20a3c2d57157c8729d4f38986ae13 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4270: [HUDI-2811] Support Spark 3.2

2021-12-12 Thread GitBox
hudi-bot removed a comment on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-991857145 ## CI report: * e515cb1eadc04d0c5d04cf170e4af4724f5bd336 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot removed a comment on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
hudi-bot removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991873871 ## CI report: * fb90953569c20a3c2d57157c8729d4f38986ae13 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4270: [HUDI-2811] Support Spark 3.2

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-991874203 ## CI report: * e515cb1eadc04d0c5d04cf170e4af4724f5bd336 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot commented on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991874214 ## CI report: * fb90953569c20a3c2d57157c8729d4f38986ae13 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4270: [HUDI-2811] Support Spark 3.2

2021-12-12 Thread GitBox
hudi-bot removed a comment on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-991873863 ## CI report: * e515cb1eadc04d0c5d04cf170e4af4724f5bd336 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot removed a comment on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
hudi-bot removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991874214 ## CI report: * fb90953569c20a3c2d57157c8729d4f38986ae13 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991875168 ## CI report: * fb90953569c20a3c2d57157c8729d4f38986ae13 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
hudi-bot removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991875168 ## CI report: * fb90953569c20a3c2d57157c8729d4f38986ae13 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4270: [HUDI-2811] Support Spark 3.2

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-991880021 ## CI report: * deafbb86c27d893f0ea6aa926a70e8a35a8fc077 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot commented on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991880035 ## CI report: * fb90953569c20a3c2d57157c8729d4f38986ae13 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4270: [HUDI-2811] Support Spark 3.2

2021-12-12 Thread GitBox
hudi-bot removed a comment on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-991874203 ## CI report: * e515cb1eadc04d0c5d04cf170e4af4724f5bd336 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] XuQianJin-Stars commented on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
XuQianJin-Stars commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991880509 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[GitHub] [hudi] XuQianJin-Stars removed a comment on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
XuQianJin-Stars removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991880509 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] XuQianJin-Stars commented on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
XuQianJin-Stars commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991880748 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] XuQianJin-Stars removed a comment on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
XuQianJin-Stars removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991880748 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [hudi] XuQianJin-Stars commented on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
XuQianJin-Stars commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991880904 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [hudi] XuQianJin-Stars removed a comment on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
XuQianJin-Stars removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991880904 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] hudi-bot removed a comment on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
hudi-bot removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991880035 ## CI report: * fb90953569c20a3c2d57157c8729d4f38986ae13 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991882154 ## CI report: * fb90953569c20a3c2d57157c8729d4f38986ae13 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
hudi-bot removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991882154 ## CI report: * fb90953569c20a3c2d57157c8729d4f38986ae13 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991883760 ## CI report: * fb90953569c20a3c2d57157c8729d4f38986ae13 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
hudi-bot removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991883760 ## CI report: * fb90953569c20a3c2d57157c8729d4f38986ae13 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991884141 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN * 929ef3884e10c34aec9b5910ed60ce5035f56187 Azure

[GitHub] [hudi] hudi-bot removed a comment on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
hudi-bot removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991884141 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN * 929ef3884e10c34aec9b5910ed60ce5035f561

[GitHub] [hudi] hudi-bot commented on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991891116 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN * 929ef3884e10c34aec9b5910ed60ce5035f56187 Azure

[GitHub] [hudi] hudi-bot removed a comment on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
hudi-bot removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991891116 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN * 929ef3884e10c34aec9b5910ed60ce5035f561

[GitHub] [hudi] hudi-bot commented on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991891522 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN * 929ef3884e10c34aec9b5910ed60ce5035f56187 Azure

[GitHub] [hudi] zhangyue19921010 edited a comment on issue #4275: [SUPPORT] How can I control the number of archive files

2021-12-12 Thread GitBox
zhangyue19921010 edited a comment on issue #4275: URL: https://github.com/apache/hudi/issues/4275#issuecomment-991418197 Hi @JoshuaZhuCN , I also meet this issue when using S3. Maybe the storage you used is not support append action. So that each time archive will create a new archive fi

[GitHub] [hudi] fengjian428 commented on issue #3755: [Delta Streamer] file name mismatch with meta when compaction running

2021-12-12 Thread GitBox
fengjian428 commented on issue #3755: URL: https://github.com/apache/hudi/issues/3755#issuecomment-991894043 > We made some fix around illegal arg exception you have posted above. Can you try with 0.10.0. Also, interested to understand more details around "after running few hours, the comp

[GitHub] [hudi] YannByron commented on pull request #4270: [HUDI-2811] Support Spark 3.2

2021-12-12 Thread GitBox
YannByron commented on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-991897465 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [hudi] hudi-bot commented on pull request #4270: [HUDI-2811] Support Spark 3.2

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-991897517 ## CI report: * deafbb86c27d893f0ea6aa926a70e8a35a8fc077 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4270: [HUDI-2811] Support Spark 3.2

2021-12-12 Thread GitBox
hudi-bot removed a comment on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-991880021 ## CI report: * deafbb86c27d893f0ea6aa926a70e8a35a8fc077 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991897533 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN * f5602d4c7e622973626effc61b831b36125234fd UNKNO

[GitHub] [hudi] hudi-bot removed a comment on pull request #4291: [HUDI-2990] Delete partitions without metadata sync to hms

2021-12-12 Thread GitBox
hudi-bot removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-991891522 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN * 929ef3884e10c34aec9b5910ed60ce5035f561

[GitHub] [hudi] JoshuaZhuCN commented on issue #3981: [SUPPORT] If the HUDI table(MOR) contains only log files, the Spark Datasource cannot obtain data in snapshot mode

2021-12-12 Thread GitBox
JoshuaZhuCN commented on issue #3981: URL: https://github.com/apache/hudi/issues/3981#issuecomment-991898766 > thanks for confirming this. Can we close it out if you don't have more questions This issue can be closed, thanks for answering my doubts. -- This is an automated message

[GitHub] [hudi] hudi-bot commented on pull request #4270: [HUDI-2811] Support Spark 3.2

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-991904382 ## CI report: * deafbb86c27d893f0ea6aa926a70e8a35a8fc077 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4270: [HUDI-2811] Support Spark 3.2

2021-12-12 Thread GitBox
hudi-bot removed a comment on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-991897517 ## CI report: * deafbb86c27d893f0ea6aa926a70e8a35a8fc077 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] nsivabalan commented on issue #3418: [SUPPORT] Hudi Upsert Very Slow/ Failed With No Space Left on Device

2021-12-12 Thread GitBox
nsivabalan commented on issue #3418: URL: https://github.com/apache/hudi/issues/3418#issuecomment-991913109 this is not reproducible locally. Only diff I see is GLOBAL_BLOOM. I will try to test it out and let you know. in the mean time, if you can provide us with reproducible steps, would

[GitHub] [hudi] nsivabalan commented on issue #3395: [SUPPORT] Issues with read optimized query MOR table

2021-12-12 Thread GitBox
nsivabalan commented on issue #3395: URL: https://github.com/apache/hudi/issues/3395#issuecomment-991913226 thanks for the update. closing the issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [hudi] nsivabalan closed issue #3395: [SUPPORT] Issues with read optimized query MOR table

2021-12-12 Thread GitBox
nsivabalan closed issue #3395: URL: https://github.com/apache/hudi/issues/3395 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[GitHub] [hudi] nsivabalan commented on issue #3394: [SUPPORT] Question on hudi's default behaviour for UPSERT

2021-12-12 Thread GitBox
nsivabalan commented on issue #3394: URL: https://github.com/apache/hudi/issues/3394#issuecomment-991913326 if you are good, can we close the issue out please. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] nsivabalan commented on issue #3394: [SUPPORT] Question on hudi's default behaviour for UPSERT

2021-12-12 Thread GitBox
nsivabalan commented on issue #3394: URL: https://github.com/apache/hudi/issues/3394#issuecomment-991913445 feel free to reopen if need be. would be happy to help. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[GitHub] [hudi] nsivabalan closed issue #3394: [SUPPORT] Question on hudi's default behaviour for UPSERT

2021-12-12 Thread GitBox
nsivabalan closed issue #3394: URL: https://github.com/apache/hudi/issues/3394 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[GitHub] [hudi] nsivabalan commented on issue #3297: HoodieMetadataException throwed when execute merge and hoodie.metadata.enable='true'

2021-12-12 Thread GitBox
nsivabalan commented on issue #3297: URL: https://github.com/apache/hudi/issues/3297#issuecomment-991913612 Closing this out as we could not reproduce. Also, we have made lot of fixes around metadata table with 0.10.0. Please do give it a try. Feel free to re-open if you are still havin

[GitHub] [hudi] nsivabalan closed issue #3297: HoodieMetadataException throwed when execute merge and hoodie.metadata.enable='true'

2021-12-12 Thread GitBox
nsivabalan closed issue #3297: URL: https://github.com/apache/hudi/issues/3297 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[GitHub] [hudi] nsivabalan commented on issue #3242: Hive sync error by using run_sync_tool.sh JDODataStoreException: Error executing SQL query "select "DB_ID" from "DBS""

2021-12-12 Thread GitBox
nsivabalan commented on issue #3242: URL: https://github.com/apache/hudi/issues/3242#issuecomment-991913855 @niloo-sh : Can you try out what Sagar have suggested above and let us know how it goes. Or feel free to close the issue if the solution worked. -- This is an automated message fr

[GitHub] [hudi] nsivabalan commented on issue #3191: [SUPPORT]client spark-submit cmd error:Caused by: java.lang.NoSuchMethodError: org.apache.spark.sql.execution.datasources.DataSourceUtils$.PARTITIO

2021-12-12 Thread GitBox
nsivabalan commented on issue #3191: URL: https://github.com/apache/hudi/issues/3191#issuecomment-991914020 @xer001 @mdz-doit : Can you please check Raymond's comment above and respond. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hudi] nsivabalan commented on issue #3132: [SUPPORT]- HUDI table returns no records when queried using Hive with partition columns in where clause

2021-12-12 Thread GitBox
nsivabalan commented on issue #3132: URL: https://github.com/apache/hudi/issues/3132#issuecomment-991914238 @deep-teliacompany : would appreciate if you can respond so that we can investigate further. -- This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [hudi] nsivabalan commented on issue #2992: [SUPPORT] Insert_Override Api not working as expected in Hudi 0.7.0

2021-12-12 Thread GitBox
nsivabalan commented on issue #2992: URL: https://github.com/apache/hudi/issues/2992#issuecomment-991914495 @ayush71994 : Can you respond with your latest when you get a chance. would like to get to the bottom of this. -- This is an automated message from the Apache Git Service. To resp

[GitHub] [hudi] nsivabalan commented on issue #2948: [SUPPORT] EMR (5.33.x) Presto Query on _rt tables

2021-12-12 Thread GitBox
nsivabalan commented on issue #2948: URL: https://github.com/apache/hudi/issues/2948#issuecomment-991914690 @tandonraghav @veenaypatil : Can you try 0.10.0 and let us know if you issue still persists. above patch should have fixed the issue raised. -- This is an automated message from t

[GitHub] [hudi] nsivabalan commented on issue #2802: Hive read issues when different partition have different schemas.

2021-12-12 Thread GitBox
nsivabalan commented on issue #2802: URL: https://github.com/apache/hudi/issues/2802#issuecomment-991915386 @aditiwari01 : sorry for dropping the ball on this. Can you put up the patch with the fix you have locally. we can help review and land it. -- This is an automated message from t

[GitHub] [hudi] nsivabalan commented on issue #2774: [SUPPORT] NoSuchMethodError on exposing metrics

2021-12-12 Thread GitBox
nsivabalan commented on issue #2774: URL: https://github.com/apache/hudi/issues/2774#issuecomment-991915728 @vobukh : Closing the ticket as the proposed patch has been landed in master and is part of latest release. Please re-open this github issue, if the problem still persists. would be

[GitHub] [hudi] nsivabalan closed issue #2774: [SUPPORT] NoSuchMethodError on exposing metrics

2021-12-12 Thread GitBox
nsivabalan closed issue #2774: URL: https://github.com/apache/hudi/issues/2774 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[GitHub] [hudi] nsivabalan commented on issue #2688: [SUPPORT] Sync to Hive using Metastore

2021-12-12 Thread GitBox
nsivabalan commented on issue #2688: URL: https://github.com/apache/hudi/issues/2688#issuecomment-991916082 Hey folks, if you can confirm with 0.10.0, hive sync works w/ hms mode, we can close this ticket out. let me know. -- This is an automated message from the Apache Git Service. To r

[GitHub] [hudi] nsivabalan commented on issue #2529: [SUPPORT] - Hudi Jar update in EMR

2021-12-12 Thread GitBox
nsivabalan commented on issue #2529: URL: https://github.com/apache/hudi/issues/2529#issuecomment-991916302 CCn @kywe665 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] nsivabalan commented on issue #2529: [SUPPORT] - Hudi Jar update in EMR

2021-12-12 Thread GitBox
nsivabalan commented on issue #2529: URL: https://github.com/apache/hudi/issues/2529#issuecomment-991916396 @bhasudha @kywe665 : Once we add the faq entry, feel free to close the ticket. -- This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [hudi] nsivabalan commented on issue #2439: [SUPPORT] Unable to sync with external hive metastore via metastore uris in the thrift protocol

2021-12-12 Thread GitBox
nsivabalan commented on issue #2439: URL: https://github.com/apache/hudi/issues/2439#issuecomment-991918349 @rubenssoto : can you confirm that all connection configs are intact in your set up. these are the ones that worked for sagar. "javax.jdo.option.ConnectionURL": "jdbc

[GitHub] [hudi] manojpec opened a new pull request #4292: [HUDI-2763] Tests for Metadata table records key deduplication

2021-12-12 Thread GitBox
manojpec opened a new pull request #4292: URL: https://github.com/apache/hudi/pull/4292 ## What is the purpose of the pull request Tests for Metadata table records key deduplication feature ## Brief change log Added the following on top of the Parent PR: https://github.

[GitHub] [hudi] hudi-bot commented on pull request #4292: [HUDI-2763] Tests for Metadata table records key deduplication

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4292: URL: https://github.com/apache/hudi/pull/4292#issuecomment-991934060 ## CI report: * cae968437fec070a998706f34b78b0c5592c6064 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[GitHub] [hudi] hudi-bot commented on pull request #4292: [HUDI-2763] Tests for Metadata table records key deduplication

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4292: URL: https://github.com/apache/hudi/pull/4292#issuecomment-991934546 ## CI report: * cae968437fec070a998706f34b78b0c5592c6064 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4292: [HUDI-2763] Tests for Metadata table records key deduplication

2021-12-12 Thread GitBox
hudi-bot removed a comment on pull request #4292: URL: https://github.com/apache/hudi/pull/4292#issuecomment-991934060 ## CI report: * cae968437fec070a998706f34b78b0c5592c6064 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4292: [HUDI-2763] Tests for Metadata table records key deduplication

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4292: URL: https://github.com/apache/hudi/pull/4292#issuecomment-991942648 ## CI report: * cae968437fec070a998706f34b78b0c5592c6064 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4292: [HUDI-2763] Tests for Metadata table records key deduplication

2021-12-12 Thread GitBox
hudi-bot removed a comment on pull request #4292: URL: https://github.com/apache/hudi/pull/4292#issuecomment-991934546 ## CI report: * cae968437fec070a998706f34b78b0c5592c6064 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] manojpec opened a new pull request #4293: [HUDI-2981][HUDI-2982] Metadata table - enabling virtual keys and key deduplication by default

2021-12-12 Thread GitBox
manojpec opened a new pull request #4293: URL: https://github.com/apache/hudi/pull/4293 ## What is the purpose of the pull request - Enabling virtual keys for metadata table. Meta fields like hoodie_record_key, hoodie_commit_time, etc., will not be added to metadata table rec

[jira] [Updated] (HUDI-2982) Verify Metadata table key deduplication and enable by default

2021-12-12 Thread Manoj Govindassamy (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HUDI-2982: - Status: Patch Available (was: In Progress) > Verify Metadata table key deduplication and

[jira] [Updated] (HUDI-2981) Verify Metadata table virtual keys support and enable by default

2021-12-12 Thread Manoj Govindassamy (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HUDI-2981: - Status: Patch Available (was: In Progress) > Verify Metadata table virtual keys support a

[jira] [Updated] (HUDI-2981) Verify Metadata table virtual keys support and enable by default

2021-12-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2981: - Labels: pull-request-available (was: ) > Verify Metadata table virtual keys support and enable by

[GitHub] [hudi] hudi-bot commented on pull request #4293: [HUDI-2981][HUDI-2982] Metadata table - enabling virtual keys and key deduplication by default

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4293: URL: https://github.com/apache/hudi/pull/4293#issuecomment-991958784 ## CI report: * 3c75b5af15f36dcc8ea7b0dbb9408134cca13c82 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[GitHub] [hudi] hudi-bot commented on pull request #4293: [HUDI-2981][HUDI-2982] Metadata table - enabling virtual keys and key deduplication by default

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4293: URL: https://github.com/apache/hudi/pull/4293#issuecomment-991959232 ## CI report: * 3c75b5af15f36dcc8ea7b0dbb9408134cca13c82 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4293: [HUDI-2981][HUDI-2982] Metadata table - enabling virtual keys and key deduplication by default

2021-12-12 Thread GitBox
hudi-bot removed a comment on pull request #4293: URL: https://github.com/apache/hudi/pull/4293#issuecomment-991958784 ## CI report: * 3c75b5af15f36dcc8ea7b0dbb9408134cca13c82 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] manojpec commented on pull request #4067: [HUDI-2763] Metadata table records key deduplication

2021-12-12 Thread GitBox
manojpec commented on pull request #4067: URL: https://github.com/apache/hudi/pull/4067#issuecomment-991963161 > @manojpec : Can you push this to another branch of yours by enabling the new config. Wanna see if CI suceeds w/o any issues. Sure, here is the PR https://github.com/apache

[GitHub] [hudi] manojpec commented on a change in pull request #4067: [HUDI-2763] Metadata table records key deduplication

2021-12-12 Thread GitBox
manojpec commented on a change in pull request #4067: URL: https://github.com/apache/hudi/pull/4067#discussion_r767238137 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieLogBlockFactory.java ## @@ -0,0 +1,90 @@ +/* + * Licensed to the Apach

[GitHub] [hudi] hudi-bot removed a comment on pull request #4293: [HUDI-2981][HUDI-2982] Metadata table - enabling virtual keys and key deduplication by default

2021-12-12 Thread GitBox
hudi-bot removed a comment on pull request #4293: URL: https://github.com/apache/hudi/pull/4293#issuecomment-991959232 ## CI report: * 3c75b5af15f36dcc8ea7b0dbb9408134cca13c82 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4293: [HUDI-2981][HUDI-2982] Metadata table - enabling virtual keys and key deduplication by default

2021-12-12 Thread GitBox
hudi-bot commented on pull request #4293: URL: https://github.com/apache/hudi/pull/4293#issuecomment-991965778 ## CI report: * 3c75b5af15f36dcc8ea7b0dbb9408134cca13c82 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] waywtdcc commented on issue #4249: [SUPPORT]FLINK CDC WRITE HUDI, restart job get exception:org.apache.hudi.org.apache.avro.InvalidAvroMagicException: Not an Avro data file

2021-12-12 Thread GitBox
waywtdcc commented on issue #4249: URL: https://github.com/apache/hudi/issues/4249#issuecomment-991997134 I also encountered this problem. Is this a bug？ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [hudi] waywtdcc edited a comment on issue #4249: [SUPPORT]FLINK CDC WRITE HUDI, restart job get exception:org.apache.hudi.org.apache.avro.InvalidAvroMagicException: Not an Avro data file

2021-12-12 Thread GitBox
waywtdcc edited a comment on issue #4249: URL: https://github.com/apache/hudi/issues/4249#issuecomment-991997134 I also encountered this problem. Is this a bug. What version of Hudi is this? I encountered this problem in 0.9 -- This is an automated message from the Apache Git Service. To

  1   2   3   >