[GitHub] [hudi] dongkelun opened a new pull request #5051: [Hudi-3643] Fix hive count exception when the table is empty and the path depth is less than 3

dongkelun opened a new pull request #5051: URL: https://github.com/apache/hudi/pull/5051 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpo

[jira] [Updated] (HUDI-3643) Hive count throws exception when the table is empty and the path depth is less than 3

[ https://issues.apache.org/jira/browse/HUDI-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 董可伦 updated HUDI-3643: -- Description: java.lang.NullPointerException     at org.apache.hudi.hadoop.utils.HoodieInputFormatUtils.getTableMetaClie

[GitHub] [hudi] hudi-bot commented on pull request #5051: [Hudi-3643] Fix hive count exception when the table is empty and the path depth is less than 3

hudi-bot commented on pull request #5051: URL: https://github.com/apache/hudi/pull/5051#issuecomment-1068801861 ## CI report: * d680cedeeb4e5bfa8cc9acc6e56834cc73c1199c UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot commented on pull request #5051: [Hudi-3643] Fix hive count exception when the table is empty and the path depth is less than 3

hudi-bot commented on pull request #5051: URL: https://github.com/apache/hudi/pull/5051#issuecomment-1068803658 ## CI report: * d680cedeeb4e5bfa8cc9acc6e56834cc73c1199c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #5051: [Hudi-3643] Fix hive count exception when the table is empty and the path depth is less than 3

hudi-bot removed a comment on pull request #5051: URL: https://github.com/apache/hudi/pull/5051#issuecomment-1068801861 ## CI report: * d680cedeeb4e5bfa8cc9acc6e56834cc73c1199c UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4918: [HUDI-3518] Make HiveSchemaProvider support AWS Glue Catalog

hudi-bot commented on pull request #4918: URL: https://github.com/apache/hudi/pull/4918#issuecomment-1068806989 ## CI report: * 4be9c06e21020e9419112cc5402f3b2455c90c73 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4918: [HUDI-3518] Make HiveSchemaProvider support AWS Glue Catalog

hudi-bot removed a comment on pull request #4918: URL: https://github.com/apache/hudi/pull/4918#issuecomment-1068770674 ## CI report: * 4be9c06e21020e9419112cc5402f3b2455c90c73 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4752: [WIP][HUDI-3088] Use Spark 3.2 as default Spark version

hudi-bot commented on pull request #4752: URL: https://github.com/apache/hudi/pull/4752#issuecomment-1068808666 ## CI report: * e04440e74c4b840224acd27aebd7be8b53dd20a2 UNKNOWN * 5562250d34698576802a8d77ddf70ad77dfbcd88 UNKNOWN * 935d2ad75c650d334b19d4ac1a48255fc929bf92 UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4752: [WIP][HUDI-3088] Use Spark 3.2 as default Spark version

hudi-bot removed a comment on pull request #4752: URL: https://github.com/apache/hudi/pull/4752#issuecomment-1068778693 ## CI report: * e04440e74c4b840224acd27aebd7be8b53dd20a2 UNKNOWN * 5562250d34698576802a8d77ddf70ad77dfbcd88 UNKNOWN * 935d2ad75c650d334b19d4ac1a48255fc929b

[jira] [Created] (HUDI-3644) hoodie log scan bug cause data duplication

hd zhou created HUDI-3644: - Summary: hoodie log scan bug cause data duplication Key: HUDI-3644 URL: https://issues.apache.org/jira/browse/HUDI-3644 Project: Apache Hudi Issue Type: Bug Re

[GitHub] [hudi] hudi-bot commented on pull request #5042: Three bulk_insert files are concurrently submitted and executed with a difference of 2s, the insert fails occasionally.

hudi-bot commented on pull request #5042: URL: https://github.com/apache/hudi/pull/5042#issuecomment-1068826183 ## CI report: * 7ee24be4d11864af37bf300250d571e15d5f9ae9 UNKNOWN * 107dab60f6541702644f4c87b7e92ba68f2fe6ef Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #5042: Three bulk_insert files are concurrently submitted and executed with a difference of 2s, the insert fails occasionally.

hudi-bot removed a comment on pull request #5042: URL: https://github.com/apache/hudi/pull/5042#issuecomment-1068774042 ## CI report: * 7ee24be4d11864af37bf300250d571e15d5f9ae9 UNKNOWN * Unknown: [CANCELED](TBD) * 107dab60f6541702644f4c87b7e92ba68f2fe6ef Azure: [PENDING](

[GitHub] [hudi] peanut-chenzhong commented on pull request #5042: Three bulk_insert files are concurrently submitted and executed with a difference of 2s, the insert fails occasionally.

peanut-chenzhong commented on pull request #5042: URL: https://github.com/apache/hudi/pull/5042#issuecomment-1068832023 @nsivabalan hi, test case added but seems some flink UT case failed by other changes. -- This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [hudi] peanut-chenzhong removed a comment on pull request #5042: Three bulk_insert files are concurrently submitted and executed with a difference of 2s, the insert fails occasionally.

peanut-chenzhong removed a comment on pull request #5042: URL: https://github.com/apache/hudi/pull/5042#issuecomment-1068832023 @nsivabalan hi, test case added but seems some flink UT case failed by other changes. -- This is an automated message from the Apache Git Service. To respond t

[GitHub] [hudi] peanut-chenzhong commented on pull request #5042: Three bulk_insert files are concurrently submitted and executed with a difference of 2s, the insert fails occasionally.

peanut-chenzhong commented on pull request #5042: URL: https://github.com/apache/hudi/pull/5042#issuecomment-1068834410 > @nsivabalan hi, test case added but seems some flink UT case failed by other changes. -- This is an automated message from the Apache Git Service. To

[GitHub] [hudi] hudi-bot removed a comment on pull request #5049: [HUDI-3598] row data to hoodie record map operator need always use the input operator parallelism to chained with source operator

hudi-bot removed a comment on pull request #5049: URL: https://github.com/apache/hudi/pull/5049#issuecomment-1068791746 ## CI report: * 4dfdad00bdf14e922ccbea542b7a4eaf8253cea0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #5049: [HUDI-3598] row data to hoodie record map operator need always use the input operator parallelism to chained with source operator

hudi-bot commented on pull request #5049: URL: https://github.com/apache/hudi/pull/5049#issuecomment-1068842288 ## CI report: * 37e4d237eb6aa59ec75c820f48fb40686840e6bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #5049: [HUDI-3598] Row Data to Hoodie Record Operator parallelism needs to always be consistent with input operator for chaining purpose

hudi-bot commented on pull request #5049: URL: https://github.com/apache/hudi/pull/5049#issuecomment-1068854690 ## CI report: * 37e4d237eb6aa59ec75c820f48fb40686840e6bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #5049: [HUDI-3598] Row Data to Hoodie Record Operator parallelism needs to always be consistent with input operator for chaining purpose

hudi-bot removed a comment on pull request #5049: URL: https://github.com/apache/hudi/pull/5049#issuecomment-1068842288 ## CI report: * 37e4d237eb6aa59ec75c820f48fb40686840e6bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #5049: [HUDI-3598] Row Data to Hoodie Record Operator parallelism needs to always be consistent with input operator for chaining purpose

hudi-bot commented on pull request #5049: URL: https://github.com/apache/hudi/pull/5049#issuecomment-1068857145 ## CI report: * 37e4d237eb6aa59ec75c820f48fb40686840e6bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #5049: [HUDI-3598] Row Data to Hoodie Record Operator parallelism needs to always be consistent with input operator for chaining purpose

hudi-bot removed a comment on pull request #5049: URL: https://github.com/apache/hudi/pull/5049#issuecomment-1068854690 ## CI report: * 37e4d237eb6aa59ec75c820f48fb40686840e6bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #5051: [Hudi-3643] Fix hive count exception when the table is empty and the path depth is less than 3

hudi-bot removed a comment on pull request #5051: URL: https://github.com/apache/hudi/pull/5051#issuecomment-1068803658 ## CI report: * d680cedeeb4e5bfa8cc9acc6e56834cc73c1199c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #5051: [Hudi-3643] Fix hive count exception when the table is empty and the path depth is less than 3

hudi-bot commented on pull request #5051: URL: https://github.com/apache/hudi/pull/5051#issuecomment-1068871546 ## CI report: * d680cedeeb4e5bfa8cc9acc6e56834cc73c1199c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4752: [WIP][HUDI-3088] Use Spark 3.2 as default Spark version

hudi-bot removed a comment on pull request #4752: URL: https://github.com/apache/hudi/pull/4752#issuecomment-1068808666 ## CI report: * e04440e74c4b840224acd27aebd7be8b53dd20a2 UNKNOWN * 5562250d34698576802a8d77ddf70ad77dfbcd88 UNKNOWN * 935d2ad75c650d334b19d4ac1a48255fc929b

[GitHub] [hudi] hudi-bot commented on pull request #4752: [WIP][HUDI-3088] Use Spark 3.2 as default Spark version

hudi-bot commented on pull request #4752: URL: https://github.com/apache/hudi/pull/4752#issuecomment-1068875872 ## CI report: * e04440e74c4b840224acd27aebd7be8b53dd20a2 UNKNOWN * 5562250d34698576802a8d77ddf70ad77dfbcd88 UNKNOWN * 935d2ad75c650d334b19d4ac1a48255fc929bf92 UNKN

[jira] [Created] (HUDI-3645) fix NPE caused by multiple threads accessing non-thread-safe HashMap

Jian Feng created HUDI-3645: --- Summary: fix NPE caused by multiple threads accessing non-thread-safe HashMap Key: HUDI-3645 URL: https://issues.apache.org/jira/browse/HUDI-3645 Project: Apache Hudi

[GitHub] [hudi] stym06 commented on issue #4184: [SUPPORT]parquet is not a Parquet file (too small length:4)

stym06 commented on issue #4184: URL: https://github.com/apache/hudi/issues/4184#issuecomment-1068898910 hey @danny0405 we are encountering this issue of 0-byte parquet files during presto read. probably this happened when the job was getting failed due to the OffsetOutofRangeException. PF

[GitHub] [hudi] fengjian428 commented on pull request #5028: [HUDI-3645] fix NPE caused by multiple threads accessing non-thread-safe HashMap

fengjian428 commented on pull request #5028: URL: https://github.com/apache/hudi/pull/5028#issuecomment-1068899205 > can we file a jira please https://issues.apache.org/jira/browse/HUDI-3645 -- This is an automated message from the Apache Git Service. To respond to the message, ple

[jira] [Updated] (HUDI-3645) fix NPE caused by multiple threads accessing non-thread-safe HashMap

[ https://issues.apache.org/jira/browse/HUDI-3645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3645: - Labels: pull-request-available (was: ) > fix NPE caused by multiple threads accessing non-thread

[GitHub] [hudi] hudi-bot commented on pull request #5049: [HUDI-3598] Row Data to Hoodie Record Operator parallelism needs to always be consistent with input operator for chaining purpose

hudi-bot commented on pull request #5049: URL: https://github.com/apache/hudi/pull/5049#issuecomment-1068933979 ## CI report: * 8d9fe303962ea2ebaa9081d80307d274a748fb57 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #5049: [HUDI-3598] Row Data to Hoodie Record Operator parallelism needs to always be consistent with input operator for chaining purpose

hudi-bot removed a comment on pull request #5049: URL: https://github.com/apache/hudi/pull/5049#issuecomment-1068857145 ## CI report: * 37e4d237eb6aa59ec75c820f48fb40686840e6bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] scxwhite commented on a change in pull request #5030: [HUDI-3617] MOR compact improve

scxwhite commented on a change in pull request #5030: URL: https://github.com/apache/hudi/pull/5030#discussion_r827820051 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/compact/HoodieCompactor.java ## @@ -280,8 +281,11 @@ HoodieCompacti

[GitHub] [hudi] scxwhite commented on a change in pull request #5030: [HUDI-3617] MOR compact improve

scxwhite commented on a change in pull request #5030: URL: https://github.com/apache/hudi/pull/5030#discussion_r827820051 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/compact/HoodieCompactor.java ## @@ -280,8 +281,11 @@ HoodieCompacti

[GitHub] [hudi] moritzmeister commented on issue #4939: [SUPPORT] Delta Streamer can't write to empty Hive table if .hoodie metadata directory doesn't exist

moritzmeister commented on issue #4939: URL: https://github.com/apache/hudi/issues/4939#issuecomment-1068938597 Thanks for creating the ticket! I will keep an eye on it, maybe I find some time to tackle it myself and contribute back :) -- This is an automated message from the Apache Git

[jira] [Commented] (HUDI-480) Support a querying delete data methond in incremental view

[ https://issues.apache.org/jira/browse/HUDI-480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17507460#comment-17507460 ] Nie Gus commented on HUDI-480: -- [~vinoth]  [~chenxiang]  we also face the similar issue, our

[jira] [Updated] (HUDI-3598) Row Data to Hoodie Record Operator parallelism needs to always be consistent with input operator for chaining purpose

[ https://issues.apache.org/jira/browse/HUDI-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuemeng updated HUDI-3598: -- Summary: Row Data to Hoodie Record Operator parallelism needs to always be consistent with input operator fo

[GitHub] [hudi] wangxianghu commented on pull request #5049: [HUDI-3598] Row Data to Hoodie Record Operator parallelism needs to always be consistent with input operator for chaining purpose

wangxianghu commented on pull request #5049: URL: https://github.com/apache/hudi/pull/5049#issuecomment-1069051674 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [hudi] sekaiga opened a new pull request #5052: [HUDI-3644] hoodie log scan bug cause data duplication bugfix

sekaiga opened a new pull request #5052: URL: https://github.com/apache/hudi/pull/5052 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpose

[GitHub] [hudi] hudi-bot commented on pull request #5049: [HUDI-3598] Row Data to Hoodie Record Operator parallelism needs to always be consistent with input operator for chaining purpose

hudi-bot commented on pull request #5049: URL: https://github.com/apache/hudi/pull/5049#issuecomment-1069054039 ## CI report: * 8d9fe303962ea2ebaa9081d80307d274a748fb57 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #5049: [HUDI-3598] Row Data to Hoodie Record Operator parallelism needs to always be consistent with input operator for chaining purpose

hudi-bot removed a comment on pull request #5049: URL: https://github.com/apache/hudi/pull/5049#issuecomment-1068933979 ## CI report: * 8d9fe303962ea2ebaa9081d80307d274a748fb57 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[jira] [Updated] (HUDI-3644) hoodie log scan bug cause data duplication

[ https://issues.apache.org/jira/browse/HUDI-3644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3644: - Labels: pull-request-available (was: ) > hoodie log scan bug cause data duplication > ---

[GitHub] [hudi] hudi-bot commented on pull request #5052: [HUDI-3644] hoodie log scan bug cause data duplication bugfix

hudi-bot commented on pull request #5052: URL: https://github.com/apache/hudi/pull/5052#issuecomment-1069056586 ## CI report: * 736c3427394d6c8074789c6ad7d9f0521e599a74 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot removed a comment on pull request #5052: [HUDI-3644] hoodie log scan bug cause data duplication bugfix

hudi-bot removed a comment on pull request #5052: URL: https://github.com/apache/hudi/pull/5052#issuecomment-1069056586 ## CI report: * 736c3427394d6c8074789c6ad7d9f0521e599a74 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #5052: [HUDI-3644] hoodie log scan bug cause data duplication bugfix

hudi-bot commented on pull request #5052: URL: https://github.com/apache/hudi/pull/5052#issuecomment-1069059202 ## CI report: * 736c3427394d6c8074789c6ad7d9f0521e599a74 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[jira] [Assigned] (HUDI-3614) Fix Flink/JavaDeleteHelper deduplication logic

[ https://issues.apache.org/jira/browse/HUDI-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-3614: Assignee: liujinhui (was: Raymond Xu) > Fix Flink/JavaDeleteHelper deduplication logic > -

[jira] [Assigned] (HUDI-3615) Replace RDD with HoodieData in HoodieFlink/JavaTable and commit executors

[ https://issues.apache.org/jira/browse/HUDI-3615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-3615: Assignee: liujinhui (was: Raymond Xu) > Replace RDD with HoodieData in HoodieFlink/JavaTable and c

[jira] [Updated] (HUDI-3411) Incorrect Record Key Field property Handling

[ https://issues.apache.org/jira/browse/HUDI-3411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3411: - Fix Version/s: 0.12.0 (was: 0.11.0) > Incorrect Record Key Field property Handling

[GitHub] [hudi] hudi-bot removed a comment on pull request #5049: [HUDI-3598] Row Data to Hoodie Record Operator parallelism needs to always be consistent with input operator for chaining purpose

hudi-bot removed a comment on pull request #5049: URL: https://github.com/apache/hudi/pull/5049#issuecomment-1069054039 ## CI report: * 8d9fe303962ea2ebaa9081d80307d274a748fb57 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #5049: [HUDI-3598] Row Data to Hoodie Record Operator parallelism needs to always be consistent with input operator for chaining purpose

hudi-bot commented on pull request #5049: URL: https://github.com/apache/hudi/pull/5049#issuecomment-1069111576 ## CI report: * 8d9fe303962ea2ebaa9081d80307d274a748fb57 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4724: [HUDI-2815] add partial overwrite payload to support partial overwrit…

hudi-bot removed a comment on pull request #4724: URL: https://github.com/apache/hudi/pull/4724#issuecomment-1062045316 ## CI report: * a86b7ff2d204900374751371cd2c8c36e76ea4b8 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4724: [HUDI-2815] add partial overwrite payload to support partial overwrit…

hudi-bot commented on pull request #4724: URL: https://github.com/apache/hudi/pull/4724#issuecomment-1069123625 ## CI report: * a86b7ff2d204900374751371cd2c8c36e76ea4b8 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] stayrascal commented on a change in pull request #4724: [HUDI-2815] add partial overwrite payload to support partial overwrit…

stayrascal commented on a change in pull request #4724: URL: https://github.com/apache/hudi/pull/4724#discussion_r828005650 ## File path: hudi-common/src/main/java/org/apache/hudi/common/model/HoodieRecordPayload.java ## @@ -58,6 +58,31 @@ default T preCombine(T oldValue, Prop

[GitHub] [hudi] hudi-bot removed a comment on pull request #4724: [HUDI-2815] add partial overwrite payload to support partial overwrit…

hudi-bot removed a comment on pull request #4724: URL: https://github.com/apache/hudi/pull/4724#issuecomment-1069123625 ## CI report: * a86b7ff2d204900374751371cd2c8c36e76ea4b8 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4724: [HUDI-2815] add partial overwrite payload to support partial overwrit…

hudi-bot commented on pull request #4724: URL: https://github.com/apache/hudi/pull/4724#issuecomment-1069126921 ## CI report: * a86b7ff2d204900374751371cd2c8c36e76ea4b8 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] bvaradar commented on a change in pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

bvaradar commented on a change in pull request #4910: URL: https://github.com/apache/hudi/pull/4910#discussion_r828000988 ## File path: hudi-common/src/main/java/org/apache/hudi/internal/schema/InternalSchema.java ## @@ -0,0 +1,292 @@ +/* + * Licensed to the Apache Software Fo

[GitHub] [hudi] xushiyan commented on a change in pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

xushiyan commented on a change in pull request #4957: URL: https://github.com/apache/hudi/pull/4957#discussion_r828014454 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/rollback/ListingBasedRollbackHelper.java ## @@ -145,6 +165,17 @@ pu

[jira] [Updated] (HUDI-3406) Rollback incorrectly relying on FS listing instead of Commit Metadata

[ https://issues.apache.org/jira/browse/HUDI-3406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3406: - Status: Patch Available (was: In Progress) > Rollback incorrectly relying on FS listing instead of Commit

[jira] [Updated] (HUDI-2866) Get Metadata table bootstrapping in Flink in parity with spark

[ https://issues.apache.org/jira/browse/HUDI-2866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2866: - Status: In Progress (was: Open) > Get Metadata table bootstrapping in Flink in parity with spark > --

[jira] [Updated] (HUDI-2866) Get Metadata table bootstrapping in Flink in parity with spark

[ https://issues.apache.org/jira/browse/HUDI-2866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2866: - Status: Patch Available (was: In Progress) > Get Metadata table bootstrapping in Flink in parity with spa

[GitHub] [hudi] worf0815 opened a new issue #5053: [SUPPORT] Pyspark with Hudi is not able to access GlueCatalog on EMR

worf0815 opened a new issue #5053: URL: https://github.com/apache/hudi/issues/5053 **Describe the problem you faced** Running pyspark on AWS EMR 6.5.0 Cluster with Hudi Enabled results in an exception when trying to access the glue catalog. **To Reproduce** Steps to rep

[jira] [Updated] (HUDI-3536) Implement Hudi DataHub sync

[ https://issues.apache.org/jira/browse/HUDI-3536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3536: - Summary: Implement Hudi DataHub sync (was: Implement commit callback to sync with DataHub) > Implement H

[GitHub] [hudi] bvaradar commented on a change in pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

bvaradar commented on a change in pull request #4910: URL: https://github.com/apache/hudi/pull/4910#discussion_r828030393 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/SparkMergeHelper.java ## @@ -77,14 +89,39 @@ public void runM

[GitHub] [hudi] bvaradar commented on a change in pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

bvaradar commented on a change in pull request #4910: URL: https://github.com/apache/hudi/pull/4910#discussion_r828026842 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/SparkMergeHelper.java ## @@ -77,14 +89,39 @@ public void runM

[GitHub] [hudi] bvaradar commented on a change in pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

bvaradar commented on a change in pull request #4910: URL: https://github.com/apache/hudi/pull/4910#discussion_r828027957 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/SparkMergeHelper.java ## @@ -77,14 +89,39 @@ public void runM

[GitHub] [hudi] hudi-bot removed a comment on pull request #5051: [Hudi-3643] Fix hive count exception when the table is empty and the path depth is less than 3

hudi-bot removed a comment on pull request #5051: URL: https://github.com/apache/hudi/pull/5051#issuecomment-1068871546 ## CI report: * d680cedeeb4e5bfa8cc9acc6e56834cc73c1199c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #5051: [Hudi-3643] Fix hive count exception when the table is empty and the path depth is less than 3

hudi-bot removed a comment on pull request #5051: URL: https://github.com/apache/hudi/pull/5051#issuecomment-1069167558 ## CI report: * d680cedeeb4e5bfa8cc9acc6e56834cc73c1199c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[jira] [Created] (HUDI-3646) The Hudi update syntax should not modify the nullability attribute of a column

Tao Meng created HUDI-3646: -- Summary: The Hudi update syntax should not modify the nullability attribute of a column Key: HUDI-3646 URL: https://issues.apache.org/jira/browse/HUDI-3646 Project: Apache Hudi

[jira] [Updated] (HUDI-3646) The Hudi update syntax should not modify the nullability attribute of a column

[ https://issues.apache.org/jira/browse/HUDI-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Meng updated HUDI-3646: --- Description: now， when we use sparksql to update hudi table, we find that  hudi will change the nullability a

[GitHub] [hudi] hudi-bot commented on pull request #5051: [Hudi-3643] Fix hive count exception when the table is empty and the path depth is less than 3

hudi-bot commented on pull request #5051: URL: https://github.com/apache/hudi/pull/5051#issuecomment-1069170702 ## CI report: * d680cedeeb4e5bfa8cc9acc6e56834cc73c1199c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #3808: [HUDI-2560] introduce id_based schema to support full schema evolution.

hudi-bot removed a comment on pull request #3808: URL: https://github.com/apache/hudi/pull/3808#issuecomment-1050601127 ## CI report: * b3958037952ed5e1240eec16efe5a42c0bdbd800 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #3808: [HUDI-2560] introduce id_based schema to support full schema evolution.

hudi-bot removed a comment on pull request #3808: URL: https://github.com/apache/hudi/pull/3808#issuecomment-1069191309 ## CI report: * b3958037952ed5e1240eec16efe5a42c0bdbd800 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #3808: [HUDI-2560] introduce id_based schema to support full schema evolution.

xiarixiaoyao commented on a change in pull request #3808: URL: https://github.com/apache/hudi/pull/3808#discussion_r828076062 ## File path: hudi-common/src/main/java/org/apache/hudi/internal/schema/io/FileBasedInternalSchemaStorageManager.java ## @@ -0,0 +1,169 @@ +/* + * Lice

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #3808: [HUDI-2560] introduce id_based schema to support full schema evolution.

xiarixiaoyao commented on a change in pull request #3808: URL: https://github.com/apache/hudi/pull/3808#discussion_r828076310 ## File path: hudi-common/src/main/java/org/apache/hudi/internal/schema/action/InternalSchemaMerger.java ## @@ -0,0 +1,151 @@ +/* + * Licensed to the A

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #3808: [HUDI-2560] introduce id_based schema to support full schema evolution.

xiarixiaoyao commented on a change in pull request #3808: URL: https://github.com/apache/hudi/pull/3808#discussion_r828077225 ## File path: hudi-common/src/main/java/org/apache/hudi/internal/schema/InternalSchema.java ## @@ -0,0 +1,281 @@ +/* + * Licensed to the Apache Softwar

[GitHub] [hudi] hudi-bot commented on pull request #3808: [HUDI-2560] introduce id_based schema to support full schema evolution.

hudi-bot commented on pull request #3808: URL: https://github.com/apache/hudi/pull/3808#issuecomment-1069194356 ## CI report: * b3958037952ed5e1240eec16efe5a42c0bdbd800 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] Rohit42 commented on issue #5050: [SUPPORT] Hudi clustering / deleting markers taking significant resources and time

Rohit42 commented on issue #5050: URL: https://github.com/apache/hudi/issues/5050#issuecomment-1069197079 https://user-images.githubusercontent.com/8977448/158614276-b5037ed4-975b-4923-ace2-3c47610fa0d4.png";> -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] xiarixiaoyao commented on pull request #3808: [HUDI-2560] introduce id_based schema to support full schema evolution.

xiarixiaoyao commented on pull request #3808: URL: https://github.com/apache/hudi/pull/3808#issuecomment-1069199465 > Nit comments. Overall looks good. Once, you are done with the changes, I will approve the diff. Looking into the Spark PR on top of this. > > @xiarixiaoyao : We need

[GitHub] [hudi] yuzhaojing commented on a change in pull request #4309: [HUDI-3016][RFC-43] Proposal to implement Compaction/Clustering Servi…

yuzhaojing commented on a change in pull request #4309: URL: https://github.com/apache/hudi/pull/4309#discussion_r828086193 ## File path: rfc/rfc-43/rfc-43.md ## @@ -0,0 +1,222 @@ + +# RFC-43: Implement Compaction/Clustering Service for Hudi + +## Proposers +- @yuzhaojing + +##

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

xiarixiaoyao commented on a change in pull request #4910: URL: https://github.com/apache/hudi/pull/4910#discussion_r828087468 ## File path: hudi-common/pom.xml ## @@ -108,6 +108,13 @@ jackson-databind + Review comment: oh sorry for forget that， let me

[GitHub] [hudi] hudi-bot removed a comment on pull request #4724: [HUDI-2815] add partial overwrite payload to support partial overwrit…

hudi-bot removed a comment on pull request #4724: URL: https://github.com/apache/hudi/pull/4724#issuecomment-1069126921 ## CI report: * a86b7ff2d204900374751371cd2c8c36e76ea4b8 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

xiarixiaoyao commented on a change in pull request #4910: URL: https://github.com/apache/hudi/pull/4910#discussion_r828087804 ## File path: hudi-common/src/main/java/org/apache/hudi/internal/schema/InternalSchema.java ## @@ -0,0 +1,292 @@ +/* + * Licensed to the Apache Softwar

[GitHub] [hudi] hudi-bot removed a comment on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

hudi-bot removed a comment on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-1068724220 ## CI report: * 9ba5c351c32f9f364c30b4bc9a814075150d9728 UNKNOWN * 6d129abe16af1cb27298326613053ecfa409d1f8 Azure: [SUCCESS](https://dev.azure.com/apache-hud

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

xiarixiaoyao commented on a change in pull request #4910: URL: https://github.com/apache/hudi/pull/4910#discussion_r828096644 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/AbstractHoodieLogRecordReader.java ## @@ -361,15 +372,38 @@ private boolean is

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

xiarixiaoyao commented on a change in pull request #4910: URL: https://github.com/apache/hudi/pull/4910#discussion_r828098543 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/SparkMergeHelper.java ## @@ -77,14 +89,39 @@ public void

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

xiarixiaoyao commented on a change in pull request #4910: URL: https://github.com/apache/hudi/pull/4910#discussion_r828101218 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/SparkMergeHelper.java ## @@ -77,14 +89,39 @@ public void

[GitHub] [hudi] hudi-bot commented on pull request #4724: [HUDI-2815] add partial overwrite payload to support partial overwrit…

hudi-bot commented on pull request #4724: URL: https://github.com/apache/hudi/pull/4724#issuecomment-1069213125 ## CI report: * d11c670e816a0a38f10e20c8cdbd8bb1cfc1afe5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

hudi-bot removed a comment on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-1069219410 ## CI report: * 9ba5c351c32f9f364c30b4bc9a814075150d9728 UNKNOWN * 6d129abe16af1cb27298326613053ecfa409d1f8 Azure: [SUCCESS](https://dev.azure.com/apache-hud

[GitHub] [hudi] yuzhaojing commented on a change in pull request #4309: [HUDI-3016][RFC-43] Proposal to implement Compaction/Clustering Servi…

yuzhaojing commented on a change in pull request #4309: URL: https://github.com/apache/hudi/pull/4309#discussion_r828106201 ## File path: rfc/rfc-43/rfc-43.md ## @@ -0,0 +1,222 @@ + +# RFC-43: Implement Compaction/Clustering Service for Hudi + +## Proposers +- @yuzhaojing + +##

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

xiarixiaoyao commented on a change in pull request #4910: URL: https://github.com/apache/hudi/pull/4910#discussion_r828115674 ## File path: hudi-common/src/main/java/org/apache/hudi/common/util/TableInternalSchemaUtils.java ## @@ -0,0 +1,140 @@ +/* + * Licensed to the Apache S

[GitHub] [hudi] yuzhaojing commented on a change in pull request #4309: [HUDI-3016][RFC-43] Proposal to implement Compaction/Clustering Servi…

yuzhaojing commented on a change in pull request #4309: URL: https://github.com/apache/hudi/pull/4309#discussion_r828114423 ## File path: rfc/rfc-43/rfc-43.md ## @@ -0,0 +1,222 @@ + +# RFC-43: Implement Compaction/Clustering Service for Hudi + +## Proposers +- @yuzhaojing + +##

[GitHub] [hudi] hudi-bot removed a comment on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

hudi-bot removed a comment on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-1069226243 ## CI report: * 9ba5c351c32f9f364c30b4bc9a814075150d9728 UNKNOWN * 6d129abe16af1cb27298326613053ecfa409d1f8 Azure: [SUCCESS](https://dev.azure.com/apache-hud

[GitHub] [hudi] xiarixiaoyao commented on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

xiarixiaoyao commented on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1069228921 @bvaradar Thank you very much for your comments i will rebase the code， and solve all comments next few days， thanks again -- This is an automated message from the Apach

[GitHub] [hudi] yuzhaojing commented on a change in pull request #4309: [HUDI-3016][RFC-43] Proposal to implement Compaction/Clustering Servi…

yuzhaojing commented on a change in pull request #4309: URL: https://github.com/apache/hudi/pull/4309#discussion_r828120504 ## File path: rfc/rfc-43/rfc-43.md ## @@ -0,0 +1,222 @@ + +# RFC-43: Implement Compaction/Clustering Service for Hudi + +## Proposers +- @yuzhaojing + +##

[GitHub] [hudi] hudi-bot removed a comment on pull request #5051: [Hudi-3643] Fix hive count exception when the table is empty and the path depth is less than 3

hudi-bot removed a comment on pull request #5051: URL: https://github.com/apache/hudi/pull/5051#issuecomment-1069170702 ## CI report: * d680cedeeb4e5bfa8cc9acc6e56834cc73c1199c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] yuzhaojing commented on a change in pull request #4309: [HUDI-3016][RFC-43] Proposal to implement Compaction/Clustering Servi…

yuzhaojing commented on a change in pull request #4309: URL: https://github.com/apache/hudi/pull/4309#discussion_r828124415 ## File path: rfc/rfc-43/rfc-43.md ## @@ -0,0 +1,222 @@ + +# RFC-43: Implement Compaction/Clustering Service for Hudi + +## Proposers +- @yuzhaojing + +##

[GitHub] [hudi] yuzhaojing commented on pull request #4309: [HUDI-3016][RFC-43] Proposal to implement Compaction/Clustering Servi…

yuzhaojing commented on pull request #4309: URL: https://github.com/apache/hudi/pull/4309#issuecomment-1069238198 @prashantwason @nsivabalan Thanks for the review, I'll be updating the RFC next week, looking forward to more comments from you. -- This is an automated message from the Apa

[GitHub] [hudi] hudi-bot commented on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

hudi-bot commented on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-1069242119 ## CI report: * 9ba5c351c32f9f364c30b4bc9a814075150d9728 UNKNOWN * 6d129abe16af1cb27298326613053ecfa409d1f8 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot commented on pull request #5051: [Hudi-3643] Fix hive count exception when the table is empty and the path depth is less than 3

hudi-bot commented on pull request #5051: URL: https://github.com/apache/hudi/pull/5051#issuecomment-1069253618 ## CI report: * 8639262558410efbb13090f02f0cb3d531b5385a Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] worf0815 opened a new issue #5054: [SUPPORT] Hudi Failing with out of memory issue on Glue with >300 Mio. Records

worf0815 opened a new issue #5054: URL: https://github.com/apache/hudi/issues/5054 **Describe the problem you faced** We are trying to ingest and deduplicate via Hudi a table with a total record size of 25 billion where each record is about 3-4kb size (there are even larger tables i

[GitHub] [hudi] hudi-bot removed a comment on pull request #3808: [HUDI-2560] introduce id_based schema to support full schema evolution.

hudi-bot removed a comment on pull request #3808: URL: https://github.com/apache/hudi/pull/3808#issuecomment-1069194356 ## CI report: * b3958037952ed5e1240eec16efe5a42c0bdbd800 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

  1   2   3   >