[GitHub] [hudi] leesf commented on a change in pull request #4222: [HUDI-2849] improve SparkUI job description for write path

2021-12-08 Thread GitBox
leesf commented on a change in pull request #4222: URL: https://github.com/apache/hudi/pull/4222#discussion_r764625760 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/DeltaSync.java ## @@ -441,9 +441,10 @@ public void refreshTimeline() throws

[GitHub] [hudi] leesf closed pull request #4193: [HUDI-2915] Fix field not found error for sparksql

2021-12-08 Thread GitBox
leesf closed pull request #4193: URL: https://github.com/apache/hudi/pull/4193 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[GitHub] [hudi] hudi-bot removed a comment on pull request #4246: [MINOR] Update DOAP with 0.10.0 Release

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4246: URL: https://github.com/apache/hudi/pull/4246#issuecomment-988561713 ## CI report: * b40dde1704cd0c69ec1981bfd411e64bd46831a4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4246: [MINOR] Update DOAP with 0.10.0 Release

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4246: URL: https://github.com/apache/hudi/pull/4246#issuecomment-988584778 ## CI report: * b40dde1704cd0c69ec1981bfd411e64bd46831a4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] YuweiXiao commented on a change in pull request #4222: [HUDI-2849] improve SparkUI job description for write path

2021-12-08 Thread GitBox
YuweiXiao commented on a change in pull request #4222: URL: https://github.com/apache/hudi/pull/4222#discussion_r764627374 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/DeltaSync.java ## @@ -441,9 +441,10 @@ public void refreshTimeline() th

[GitHub] [hudi] kywe665 opened a new pull request #4250: [HUDI-2956] - Updating write docs for deletes and full write path description

2021-12-08 Thread GitBox
kywe665 opened a new pull request #4250: URL: https://github.com/apache/hudi/pull/4250 ## What is the purpose of the pull request Added more details for deletes and described high level full write path as described in this deep dive: https://www.youtube.com/watch?v=N2eDfU_rQ_U

[jira] [Updated] (HUDI-2956) Improve Write docs

2021-12-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2956: - Labels: pull-request-available (was: ) > Improve Write docs > -- > >

[jira] [Updated] (HUDI-2956) Improve Write docs

2021-12-08 Thread Kyle Weller (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Weller updated HUDI-2956: -- Status: Patch Available (was: In Progress) > Improve Write docs > -- > >

[GitHub] [hudi] guanlisheng commented on issue #4055: [SUPPORT] Hudi with SqlQueryBasedTransformer fails-> spark error exit 134 or exit 143 in "isEmpty at DeltaSync.java:344" : Container from a bad n

2021-12-08 Thread GitBox
guanlisheng commented on issue #4055: URL: https://github.com/apache/hudi/issues/4055#issuecomment-988592379 Hi there, I have an identical issue when enabling my customized transformer class in Hudi 7.0 on EMR. the transformer class is performing `mapPartitions` operation. -- This i

[GitHub] [hudi] hudi-bot removed a comment on pull request #4245: [MINOR] remove unuse construction method

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4245: URL: https://github.com/apache/hudi/pull/4245#issuecomment-988581448 ## CI report: * 6f4e9f5fd7387cc3ec4dfa8d7f7a83a3abcbd0c0 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4245: [MINOR] remove unuse construction method

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4245: URL: https://github.com/apache/hudi/pull/4245#issuecomment-988610403 ## CI report: * 6f4e9f5fd7387cc3ec4dfa8d7f7a83a3abcbd0c0 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[jira] [Created] (HUDI-2957) Shade kryo jar for flink bundle jar

2021-12-08 Thread Danny Chen (Jira)
Danny Chen created HUDI-2957: Summary: Shade kryo jar for flink bundle jar Key: HUDI-2957 URL: https://issues.apache.org/jira/browse/HUDI-2957 Project: Apache Hudi Issue Type: Improvement

[GitHub] [hudi] hudi-bot removed a comment on pull request #4245: [MINOR] remove unuse construction method

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4245: URL: https://github.com/apache/hudi/pull/4245#issuecomment-988610403 ## CI report: * 6f4e9f5fd7387cc3ec4dfa8d7f7a83a3abcbd0c0 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4245: [MINOR] remove unuse construction method

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4245: URL: https://github.com/apache/hudi/pull/4245#issuecomment-988641881 ## CI report: * 2b1b0fcc6c35bd83846bf0914babc2199165f5c5 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] Limess commented on issue #4146: [SUPPORT] Deltastreamer commits with a custom checkpoint configuration are skipped if the generated checkpoint matches the previous commit checkpoint

2021-12-08 Thread GitBox
Limess commented on issue #4146: URL: https://github.com/apache/hudi/issues/4146#issuecomment-988642069 > @Limess : So, is the expectation that, if you set checkpoint = 0, deltastreamer should start from scratch as though we are starting deltastreamer for the first time ? Yes that's

[GitHub] [hudi] danny0405 opened a new pull request #4251: [HUDI-2957] Shade kryo jar for flink bundle jar

2021-12-08 Thread GitBox
danny0405 opened a new pull request #4251: URL: https://github.com/apache/hudi/pull/4251 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpo

[jira] [Updated] (HUDI-2957) Shade kryo jar for flink bundle jar

2021-12-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2957: - Labels: pull-request-available (was: ) > Shade kryo jar for flink bundle jar > --

[GitHub] [hudi] hudi-bot commented on pull request #4251: [HUDI-2957] Shade kryo jar for flink bundle jar

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4251: URL: https://github.com/apache/hudi/pull/4251#issuecomment-988657584 ## CI report: * 2a84f3eeef355177e32aebd62a8eb4ed2712a647 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4251: [HUDI-2957] Shade kryo jar for flink bundle jar

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4251: URL: https://github.com/apache/hudi/pull/4251#issuecomment-988657584 ## CI report: * 2a84f3eeef355177e32aebd62a8eb4ed2712a647 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4251: [HUDI-2957] Shade kryo jar for flink bundle jar

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4251: URL: https://github.com/apache/hudi/pull/4251#issuecomment-988659621 ## CI report: * 2a84f3eeef355177e32aebd62a8eb4ed2712a647 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] danny0405 commented on pull request #4246: [MINOR] Update DOAP with 0.10.0 Release

2021-12-08 Thread GitBox
danny0405 commented on pull request #4246: URL: https://github.com/apache/hudi/pull/4246#issuecomment-988661990 The test failure should not be caused by this patch, so i would just merge it. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [hudi] danny0405 merged pull request #4246: [MINOR] Update DOAP with 0.10.0 Release

2021-12-08 Thread GitBox
danny0405 merged pull request #4246: URL: https://github.com/apache/hudi/pull/4246 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[hudi] branch master updated (c9e18d1 -> c56d93e)

2021-12-08 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from c9e18d1 [HUDI-2942] add error message log in HoodieCombineHiveInputFormat (#4224) add c56d93e [MINOR] Update

[GitHub] [hudi] hudi-bot removed a comment on pull request #4251: [HUDI-2957] Shade kryo jar for flink bundle jar

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4251: URL: https://github.com/apache/hudi/pull/4251#issuecomment-988659621 ## CI report: * 2a84f3eeef355177e32aebd62a8eb4ed2712a647 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4251: [HUDI-2957] Shade kryo jar for flink bundle jar

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4251: URL: https://github.com/apache/hudi/pull/4251#issuecomment-988694462 ## CI report: * 2a84f3eeef355177e32aebd62a8eb4ed2712a647 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[jira] [Updated] (HUDI-2958) Automatically set spark.sql.parquet.writelegacyformat; When using bulkinsert to insert data which contains decimal Type.

2021-12-08 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao meng updated HUDI-2958: --- Summary: Automatically set spark.sql.parquet.writelegacyformat; When using bulkinsert to insert data which con

[jira] [Created] (HUDI-2958) Automatically set spark.sql.parquet.writelegacyformat. When using bulkinsert to insert data will contains decimal Type.

2021-12-08 Thread tao meng (Jira)
tao meng created HUDI-2958: -- Summary: Automatically set spark.sql.parquet.writelegacyformat. When using bulkinsert to insert data will contains decimal Type. Key: HUDI-2958 URL: https://issues.apache.org/jira/browse/HUDI

[jira] [Created] (HUDI-2959) Fix the thread leak of cleaning service

2021-12-08 Thread Danny Chen (Jira)
Danny Chen created HUDI-2959: Summary: Fix the thread leak of cleaning service Key: HUDI-2959 URL: https://issues.apache.org/jira/browse/HUDI-2959 Project: Apache Hudi Issue Type: Bug C

[GitHub] [hudi] danny0405 opened a new pull request #4252: [HUDI-2959] Fix the thread leak of cleaning service

2021-12-08 Thread GitBox
danny0405 opened a new pull request #4252: URL: https://github.com/apache/hudi/pull/4252 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpo

[GitHub] [hudi] danny0405 commented on pull request #4252: [HUDI-2959] Fix the thread leak of cleaning service

2021-12-08 Thread GitBox
danny0405 commented on pull request #4252: URL: https://github.com/apache/hudi/pull/4252#issuecomment-988774462 @vinothchandar , can you take a look, thanks so much ~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[jira] [Updated] (HUDI-2959) Fix the thread leak of cleaning service

2021-12-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2959: - Labels: pull-request-available (was: ) > Fix the thread leak of cleaning service > --

[GitHub] [hudi] danny0405 commented on pull request #4251: [HUDI-2957] Shade kryo jar for flink bundle jar

2021-12-08 Thread GitBox
danny0405 commented on pull request #4251: URL: https://github.com/apache/hudi/pull/4251#issuecomment-988775610 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [hudi] hudi-bot removed a comment on pull request #4251: [HUDI-2957] Shade kryo jar for flink bundle jar

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4251: URL: https://github.com/apache/hudi/pull/4251#issuecomment-988694462 ## CI report: * 2a84f3eeef355177e32aebd62a8eb4ed2712a647 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4252: [HUDI-2959] Fix the thread leak of cleaning service

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4252: URL: https://github.com/apache/hudi/pull/4252#issuecomment-988776087 ## CI report: * b16a5686bd6a8a03aa2847624fa5cf3e2e9d36ec UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[GitHub] [hudi] hudi-bot commented on pull request #4251: [HUDI-2957] Shade kryo jar for flink bundle jar

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4251: URL: https://github.com/apache/hudi/pull/4251#issuecomment-988776048 ## CI report: * 2a84f3eeef355177e32aebd62a8eb4ed2712a647 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4252: [HUDI-2959] Fix the thread leak of cleaning service

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4252: URL: https://github.com/apache/hudi/pull/4252#issuecomment-988776087 ## CI report: * b16a5686bd6a8a03aa2847624fa5cf3e2e9d36ec UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4252: [HUDI-2959] Fix the thread leak of cleaning service

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4252: URL: https://github.com/apache/hudi/pull/4252#issuecomment-98863 ## CI report: * b16a5686bd6a8a03aa2847624fa5cf3e2e9d36ec Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] xiarixiaoyao opened a new pull request #4253: [HUDI-2958] Automatically set spark.sql.parquet.writelegacyformat, when using bulkinsert to insert data which contains decimalType

2021-12-08 Thread GitBox
xiarixiaoyao opened a new pull request #4253: URL: https://github.com/apache/hudi/pull/4253 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What i

[jira] [Updated] (HUDI-2958) Automatically set spark.sql.parquet.writelegacyformat; When using bulkinsert to insert data which contains decimal Type.

2021-12-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2958: - Labels: pull-request-available (was: ) > Automatically set spark.sql.parquet.writelegacyformat; W

[GitHub] [hudi] hudi-bot commented on pull request #4252: [HUDI-2959] Fix the thread leak of cleaning service

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4252: URL: https://github.com/apache/hudi/pull/4252#issuecomment-988783594 ## CI report: * b16a5686bd6a8a03aa2847624fa5cf3e2e9d36ec Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4252: [HUDI-2959] Fix the thread leak of cleaning service

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4252: URL: https://github.com/apache/hudi/pull/4252#issuecomment-98863 ## CI report: * b16a5686bd6a8a03aa2847624fa5cf3e2e9d36ec Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4252: [HUDI-2959] Fix the thread leak of cleaning service

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4252: URL: https://github.com/apache/hudi/pull/4252#issuecomment-988787463 ## CI report: * b16a5686bd6a8a03aa2847624fa5cf3e2e9d36ec Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #4253: [HUDI-2958] Automatically set spark.sql.parquet.writelegacyformat, when using bulkinsert to insert data which contains decimalType

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4253: URL: https://github.com/apache/hudi/pull/4253#issuecomment-988787494 ## CI report: * 34dd491be3ce6d6f55627bbe3390fefbac674e8e UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4252: [HUDI-2959] Fix the thread leak of cleaning service

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4252: URL: https://github.com/apache/hudi/pull/4252#issuecomment-988783594 ## CI report: * b16a5686bd6a8a03aa2847624fa5cf3e2e9d36ec Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot removed a comment on pull request #4253: [HUDI-2958] Automatically set spark.sql.parquet.writelegacyformat, when using bulkinsert to insert data which contains decimalType

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4253: URL: https://github.com/apache/hudi/pull/4253#issuecomment-988787494 ## CI report: * 34dd491be3ce6d6f55627bbe3390fefbac674e8e UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4253: [HUDI-2958] Automatically set spark.sql.parquet.writelegacyformat, when using bulkinsert to insert data which contains decimalType

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4253: URL: https://github.com/apache/hudi/pull/4253#issuecomment-988789343 ## CI report: * 34dd491be3ce6d6f55627bbe3390fefbac674e8e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] yuzhaojing opened a new pull request #4254: [HUDI-2537] Schedule Flink compaction in service

2021-12-08 Thread GitBox
yuzhaojing opened a new pull request #4254: URL: https://github.com/apache/hudi/pull/4254 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purp

[GitHub] [hudi] hudi-bot commented on pull request #4254: [HUDI-2537] Schedule Flink compaction in service

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4254: URL: https://github.com/apache/hudi/pull/4254#issuecomment-988805075 ## CI report: * 59cdd6413be9d029f175e06e12db7893f75e7af7 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4254: [HUDI-2537] Schedule Flink compaction in service

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4254: URL: https://github.com/apache/hudi/pull/4254#issuecomment-988805075 ## CI report: * 59cdd6413be9d029f175e06e12db7893f75e7af7 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4254: [HUDI-2537] Schedule Flink compaction in service

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4254: URL: https://github.com/apache/hudi/pull/4254#issuecomment-988807064 ## CI report: * 59cdd6413be9d029f175e06e12db7893f75e7af7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] danny0405 commented on pull request #4251: [HUDI-2957] Shade kryo jar for flink bundle jar

2021-12-08 Thread GitBox
danny0405 commented on pull request #4251: URL: https://github.com/apache/hudi/pull/4251#issuecomment-988814010 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [hudi] hudi-bot removed a comment on pull request #4251: [HUDI-2957] Shade kryo jar for flink bundle jar

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4251: URL: https://github.com/apache/hudi/pull/4251#issuecomment-988776048 ## CI report: * 2a84f3eeef355177e32aebd62a8eb4ed2712a647 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4251: [HUDI-2957] Shade kryo jar for flink bundle jar

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4251: URL: https://github.com/apache/hudi/pull/4251#issuecomment-988815109 ## CI report: * 2a84f3eeef355177e32aebd62a8eb4ed2712a647 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4252: [HUDI-2959] Fix the thread leak of cleaning service

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4252: URL: https://github.com/apache/hudi/pull/4252#issuecomment-988787463 ## CI report: * b16a5686bd6a8a03aa2847624fa5cf3e2e9d36ec Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4252: [HUDI-2959] Fix the thread leak of cleaning service

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4252: URL: https://github.com/apache/hudi/pull/4252#issuecomment-988825206 ## CI report: * b16a5686bd6a8a03aa2847624fa5cf3e2e9d36ec Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #4253: [HUDI-2958] Automatically set spark.sql.parquet.writelegacyformat, when using bulkinsert to insert data which contains decimalType

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4253: URL: https://github.com/apache/hudi/pull/4253#issuecomment-988825224 ## CI report: * 34dd491be3ce6d6f55627bbe3390fefbac674e8e Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4253: [HUDI-2958] Automatically set spark.sql.parquet.writelegacyformat, when using bulkinsert to insert data which contains decimalType

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4253: URL: https://github.com/apache/hudi/pull/4253#issuecomment-988789343 ## CI report: * 34dd491be3ce6d6f55627bbe3390fefbac674e8e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot removed a comment on pull request #4020: [WIP][HUDI-2783] Upgrade HBase to 2.x

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4020: URL: https://github.com/apache/hudi/pull/4020#issuecomment-978211197 ## CI report: * 548c193ffe432033be61ca5a592f6d9760b5ebb0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4020: [WIP][HUDI-2783] Upgrade HBase to 2.x

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4020: URL: https://github.com/apache/hudi/pull/4020#issuecomment-988827020 ## CI report: * 548c193ffe432033be61ca5a592f6d9760b5ebb0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot commented on pull request #4020: [WIP][HUDI-2783] Upgrade HBase to 2.x

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4020: URL: https://github.com/apache/hudi/pull/4020#issuecomment-988828994 ## CI report: * 548c193ffe432033be61ca5a592f6d9760b5ebb0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4020: [WIP][HUDI-2783] Upgrade HBase to 2.x

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4020: URL: https://github.com/apache/hudi/pull/4020#issuecomment-988827020 ## CI report: * 548c193ffe432033be61ca5a592f6d9760b5ebb0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot removed a comment on pull request #4254: [HUDI-2537] Schedule Flink compaction in service

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4254: URL: https://github.com/apache/hudi/pull/4254#issuecomment-988807064 ## CI report: * 59cdd6413be9d029f175e06e12db7893f75e7af7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4254: [HUDI-2537] Schedule Flink compaction in service

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4254: URL: https://github.com/apache/hudi/pull/4254#issuecomment-988849669 ## CI report: * 59cdd6413be9d029f175e06e12db7893f75e7af7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4251: [HUDI-2957] Shade kryo jar for flink bundle jar

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4251: URL: https://github.com/apache/hudi/pull/4251#issuecomment-988815109 ## CI report: * 2a84f3eeef355177e32aebd62a8eb4ed2712a647 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4251: [HUDI-2957] Shade kryo jar for flink bundle jar

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4251: URL: https://github.com/apache/hudi/pull/4251#issuecomment-988861254 ## CI report: * 2a84f3eeef355177e32aebd62a8eb4ed2712a647 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] nsivabalan closed issue #2934: [SUPPORT] Parquet file does not exist when trying to read hudi table incrementally

2021-12-08 Thread GitBox
nsivabalan closed issue #2934: URL: https://github.com/apache/hudi/issues/2934 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[GitHub] [hudi] nsivabalan commented on issue #2934: [SUPPORT] Parquet file does not exist when trying to read hudi table incrementally

2021-12-08 Thread GitBox
nsivabalan commented on issue #2934: URL: https://github.com/apache/hudi/issues/2934#issuecomment-90486 Will close out the ticket as this is expected with interplays between archival and incremental queries. and since we have a patch addressing it. -- This is an automated message fr

[GitHub] [hudi] nsivabalan closed issue #3826: Deltastreamer not getting auto triggered in continuous mode

2021-12-08 Thread GitBox
nsivabalan closed issue #3826: URL: https://github.com/apache/hudi/issues/3826 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[GitHub] [hudi] nsivabalan commented on issue #3826: Deltastreamer not getting auto triggered in continuous mode

2021-12-08 Thread GitBox
nsivabalan commented on issue #3826: URL: https://github.com/apache/hudi/issues/3826#issuecomment-92722 Can you please respond. This is very common use-case and many folks in the community have been running in continuous mode. So, some env specific or config issue. Closing it for now.

[GitHub] [hudi] hudi-bot removed a comment on pull request #4252: [HUDI-2959] Fix the thread leak of cleaning service

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4252: URL: https://github.com/apache/hudi/pull/4252#issuecomment-988825206 ## CI report: * b16a5686bd6a8a03aa2847624fa5cf3e2e9d36ec Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4252: [HUDI-2959] Fix the thread leak of cleaning service

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4252: URL: https://github.com/apache/hudi/pull/4252#issuecomment-92409 ## CI report: * 15574f13d95fd781239bfb81dcd7ecef8213f6c9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] nsivabalan commented on issue #4146: [SUPPORT] Deltastreamer commits with a custom checkpoint configuration are skipped if the generated checkpoint matches the previous commit checkpoi

2021-12-08 Thread GitBox
nsivabalan commented on issue #4146: URL: https://github.com/apache/hudi/issues/4146#issuecomment-988891080 gotcha. guess the way you set the checkpoint should work based on this code block ``` if (cfg.checkpoint != null && (StringUtils.isNullOrEmpty(commitMetadata.getMetadat

[GitHub] [hudi] nsivabalan commented on issue #4146: [SUPPORT] Deltastreamer commits with a custom checkpoint configuration are skipped if the generated checkpoint matches the previous commit checkpoi

2021-12-08 Thread GitBox
nsivabalan commented on issue #4146: URL: https://github.com/apache/hudi/issues/4146#issuecomment-988892449 Can you try checkpoint = "val=0" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[jira] [Created] (HUDI-2960) create hudi table may cause memory leak in spark thrift server

2021-12-08 Thread suheng.cloud (Jira)
suheng.cloud created HUDI-2960: -- Summary: create hudi table may cause memory leak in spark thrift server Key: HUDI-2960 URL: https://issues.apache.org/jira/browse/HUDI-2960 Project: Apache Hudi

[GitHub] [hudi] Limess commented on issue #4146: [SUPPORT] Deltastreamer commits with a custom checkpoint configuration are skipped if the generated checkpoint matches the previous commit checkpoint

2021-12-08 Thread GitBox
Limess commented on issue #4146: URL: https://github.com/apache/hudi/issues/4146#issuecomment-988898544 > Can you try checkpoint = "val=0" To clarify, you're asking me to try `--checkpoint "val=0"` using the Deltastreamer CLI? -- This is an automated message from the Apache G

[GitHub] [hudi] Limess edited a comment on issue #4146: [SUPPORT] Deltastreamer commits with a custom checkpoint configuration are skipped if the generated checkpoint matches the previous commit check

2021-12-08 Thread GitBox
Limess edited a comment on issue #4146: URL: https://github.com/apache/hudi/issues/4146#issuecomment-988898544 > Can you try checkpoint = "val=0" To clarify, you're asking me to try `--checkpoint "val=0"` using the Deltastreamer CLI? Or `--checkpoint 0`? -- This is an automat

[GitHub] [hudi] Limess edited a comment on issue #4146: [SUPPORT] Deltastreamer commits with a custom checkpoint configuration are skipped if the generated checkpoint matches the previous commit check

2021-12-08 Thread GitBox
Limess edited a comment on issue #4146: URL: https://github.com/apache/hudi/issues/4146#issuecomment-988642069 > @Limess : So, is the expectation that, if you set checkpoint = 0, deltastreamer should start from scratch as though we are starting deltastreamer for the first time ? Yes

[GitHub] [hudi] hudi-bot commented on pull request #4020: [WIP][HUDI-2783] Upgrade HBase to 2.x

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4020: URL: https://github.com/apache/hudi/pull/4020#issuecomment-988899676 ## CI report: * 72ea77955da505b679945dc92ea0dd2d597bcedf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4020: [WIP][HUDI-2783] Upgrade HBase to 2.x

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4020: URL: https://github.com/apache/hudi/pull/4020#issuecomment-988828994 ## CI report: * 548c193ffe432033be61ca5a592f6d9760b5ebb0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] danny0405 commented on pull request #4252: [HUDI-2959] Fix the thread leak of cleaning service

2021-12-08 Thread GitBox
danny0405 commented on pull request #4252: URL: https://github.com/apache/hudi/pull/4252#issuecomment-988906058 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [hudi] hudi-bot commented on pull request #4252: [HUDI-2959] Fix the thread leak of cleaning service

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4252: URL: https://github.com/apache/hudi/pull/4252#issuecomment-988907016 ## CI report: * 15574f13d95fd781239bfb81dcd7ecef8213f6c9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4252: [HUDI-2959] Fix the thread leak of cleaning service

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4252: URL: https://github.com/apache/hudi/pull/4252#issuecomment-92409 ## CI report: * 15574f13d95fd781239bfb81dcd7ecef8213f6c9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot removed a comment on pull request #4020: [WIP][HUDI-2783] Upgrade HBase to 2.x

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4020: URL: https://github.com/apache/hudi/pull/4020#issuecomment-988899676 ## CI report: * 72ea77955da505b679945dc92ea0dd2d597bcedf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4020: [WIP][HUDI-2783] Upgrade HBase to 2.x

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4020: URL: https://github.com/apache/hudi/pull/4020#issuecomment-988911467 ## CI report: * 72ea77955da505b679945dc92ea0dd2d597bcedf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4020: [WIP][HUDI-2783] Upgrade HBase to 2.x

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4020: URL: https://github.com/apache/hudi/pull/4020#issuecomment-988911467 ## CI report: * 72ea77955da505b679945dc92ea0dd2d597bcedf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4020: [WIP][HUDI-2783] Upgrade HBase to 2.x

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4020: URL: https://github.com/apache/hudi/pull/4020#issuecomment-988914045 ## CI report: * 72ea77955da505b679945dc92ea0dd2d597bcedf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot commented on pull request #4252: [HUDI-2959] Fix the thread leak of cleaning service

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4252: URL: https://github.com/apache/hudi/pull/4252#issuecomment-988947837 ## CI report: * 15574f13d95fd781239bfb81dcd7ecef8213f6c9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4252: [HUDI-2959] Fix the thread leak of cleaning service

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4252: URL: https://github.com/apache/hudi/pull/4252#issuecomment-988907016 ## CI report: * 15574f13d95fd781239bfb81dcd7ecef8213f6c9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] nsivabalan commented on issue #4146: [SUPPORT] Deltastreamer commits with a custom checkpoint configuration are skipped if the generated checkpoint matches the previous commit checkpoi

2021-12-08 Thread GitBox
nsivabalan commented on issue #4146: URL: https://github.com/apache/hudi/issues/4146#issuecomment-988951531 This worked for me. First run ``` nsb$ grep "Checkpoint" /tmp/logs/log23.out 21/12/08 07:47:52 INFO DeltaSync: Checkpoint to resume from : Optional.empty 21/12/08 07:4

[GitHub] [hudi] hudi-bot removed a comment on pull request #4020: [WIP][HUDI-2783] Upgrade HBase to 2.x

2021-12-08 Thread GitBox
hudi-bot removed a comment on pull request #4020: URL: https://github.com/apache/hudi/pull/4020#issuecomment-988914045 ## CI report: * 72ea77955da505b679945dc92ea0dd2d597bcedf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4020: [WIP][HUDI-2783] Upgrade HBase to 2.x

2021-12-08 Thread GitBox
hudi-bot commented on pull request #4020: URL: https://github.com/apache/hudi/pull/4020#issuecomment-988963081 ## CI report: * a23ba86033f3215c9f57118742189ae844c6c850 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] h7kanna commented on issue #4170: [SUPPORT] Understanding Clustering Behavior

2021-12-08 Thread GitBox
h7kanna commented on issue #4170: URL: https://github.com/apache/hudi/issues/4170#issuecomment-988979839 I have hoodie.parquet.max.file.size=134217728 hoodie.parquet.small.file.limit=104857600 hoodie.clustering.plan.strategy.target.file.max.bytes=134217728 hoodie.clustering.plan

[GitHub] [hudi] h7kanna edited a comment on issue #4170: [SUPPORT] Understanding Clustering Behavior

2021-12-08 Thread GitBox
h7kanna edited a comment on issue #4170: URL: https://github.com/apache/hudi/issues/4170#issuecomment-988979839 I have hoodie.parquet.max.file.size=134217728 hoodie.parquet.small.file.limit=67108864 hoodie.clustering.plan.strategy.target.file.max.bytes=134217728 hoodie.clusterin

[GitHub] [hudi] vinothchandar commented on a change in pull request #3173: [HUDI-1951] Add bucket hash index, compatible with the hive bucket

2021-12-08 Thread GitBox
vinothchandar commented on a change in pull request #3173: URL: https://github.com/apache/hudi/pull/3173#discussion_r764911822 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/HoodieIndex.java ## @@ -122,13 +125,40 @@ public O updateLocation(O w

[GitHub] [hudi] Limess commented on issue #4146: [SUPPORT] Deltastreamer commits with a custom checkpoint configuration are skipped if the generated checkpoint matches the previous commit checkpoint

2021-12-08 Thread GitBox
Limess commented on issue #4146: URL: https://github.com/apache/hudi/issues/4146#issuecomment-989012890 We're not running in continous mode, it looks like the above might be? We're also using DFS datasource. -- This is an automated message from the Apache Git Service. To respond to the m

[jira] [Created] (HUDI-2961) Async table services can race with metadata table updates

2021-12-08 Thread Manoj Govindassamy (Jira)
Manoj Govindassamy created HUDI-2961: Summary: Async table services can race with metadata table updates Key: HUDI-2961 URL: https://issues.apache.org/jira/browse/HUDI-2961 Project: Apache Hudi

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-08 Thread GitBox
alexeykudinkin commented on a change in pull request #4178: URL: https://github.com/apache/hudi/pull/4178#discussion_r765092480 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/clustering/run/strategy/MultipleSparkJobExecutionStrategy.java ## @@

[GitHub] [hudi] alexeykudinkin commented on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-08 Thread GitBox
alexeykudinkin commented on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-989038686 Great catch @xiarixiaoyao! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[jira] [Created] (HUDI-2962) Enable metadata table along with JVM local lock provider

2021-12-08 Thread Manoj Govindassamy (Jira)
Manoj Govindassamy created HUDI-2962: Summary: Enable metadata table along with JVM local lock provider Key: HUDI-2962 URL: https://issues.apache.org/jira/browse/HUDI-2962 Project: Apache Hudi

[jira] [Created] (HUDI-2963) Update configs for 0.10.0

2021-12-08 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-2963: - Summary: Update configs for 0.10.0 Key: HUDI-2963 URL: https://issues.apache.org/jira/browse/HUDI-2963 Project: Apache Hudi Issue Type: Improvemen

  1   2   3   4   >