[GitHub] [hudi] hudi-bot removed a comment on pull request #4752: [WIP][HUDI-3088] Use Spark 3.2 as default Spark version

hudi-bot removed a comment on pull request #4752: URL: https://github.com/apache/hudi/pull/4752#issuecomment-1035952410 ## CI report: * bc24e231f7f7e86f0eeac7716f0b1e9293374169 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4752: [WIP][HUDI-3088] Use Spark 3.2 as default Spark version

hudi-bot commented on pull request #4752: URL: https://github.com/apache/hudi/pull/4752#issuecomment-1035961795 ## CI report: * bc24e231f7f7e86f0eeac7716f0b1e9293374169 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] codope commented on issue #3868: [SUPPORT] hive syncing with `--spark-datasource` (first title was: Querying hudi datasets from standalone metastore)

codope commented on issue #3868: URL: https://github.com/apache/hudi/issues/3868#issuecomment-1035963670 @matthiasdg Could you remove the partition extraction config (by default it is slash encoded day partition) and try again? I have updated the gist with both failed and successful runs i

[GitHub] [hudi] danny0405 commented on a change in pull request #4753: [HUDI-3370] The files recorded in the commit may not match the actual ones for MOR Compaction

danny0405 commented on a change in pull request #4753: URL: https://github.com/apache/hudi/pull/4753#discussion_r804430279 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/HoodieSparkCompactor.java ## @@ -43,19 +48,24 @@ public HoodieSparkCompac

[GitHub] [hudi] danny0405 commented on a change in pull request #4753: [HUDI-3370] The files recorded in the commit may not match the actual ones for MOR Compaction

danny0405 commented on a change in pull request #4753: URL: https://github.com/apache/hudi/pull/4753#discussion_r804432099 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieCompactor.java ## @@ -293,4 +297,17 @@ private String getSchemaFromLatestInstan

[GitHub] [hudi] danny0405 commented on a change in pull request #4791: [HUDI-3412] TypedProperties no need to create new set when check key …

danny0405 commented on a change in pull request #4791: URL: https://github.com/apache/hudi/pull/4791#discussion_r804433456 ## File path: hudi-common/src/main/java/org/apache/hudi/common/config/TypedProperties.java ## @@ -44,27 +43,22 @@ public TypedProperties(Properties defaul

[GitHub] [hudi] hudi-bot commented on pull request #4752: [WIP][HUDI-3088] Use Spark 3.2 as default Spark version

hudi-bot commented on pull request #4752: URL: https://github.com/apache/hudi/pull/4752#issuecomment-1035971918 ## CI report: * bc24e231f7f7e86f0eeac7716f0b1e9293374169 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #4752: [WIP][HUDI-3088] Use Spark 3.2 as default Spark version

hudi-bot removed a comment on pull request #4752: URL: https://github.com/apache/hudi/pull/4752#issuecomment-1035961795 ## CI report: * bc24e231f7f7e86f0eeac7716f0b1e9293374169 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] yihua opened a new pull request #4792: [HUDI-3366] Remove hardcoded logic of disabling metadata table in tests

yihua opened a new pull request #4792: URL: https://github.com/apache/hudi/pull/4792 ## What is the purpose of the pull request This PR removes hardcoded logic of disabling metadata table in tests and fixes test that are broken with metadata table. ## Verify this pull request

[jira] [Updated] (HUDI-3366) Remove unnecessary hardcoded logic of disabling metadata table in tests

[ https://issues.apache.org/jira/browse/HUDI-3366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3366: - Labels: pull-request-available (was: ) > Remove unnecessary hardcoded logic of disabling metadata

[GitHub] [hudi] hudi-bot commented on pull request #4792: [HUDI-3366] Remove hardcoded logic of disabling metadata table in tests

hudi-bot commented on pull request #4792: URL: https://github.com/apache/hudi/pull/4792#issuecomment-1035979794 ## CI report: * 1b2bdd81dc3f7edea2934b1cd461ee50dff029b7 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot removed a comment on pull request #4792: [HUDI-3366] Remove hardcoded logic of disabling metadata table in tests

hudi-bot removed a comment on pull request #4792: URL: https://github.com/apache/hudi/pull/4792#issuecomment-1035979794 ## CI report: * 1b2bdd81dc3f7edea2934b1cd461ee50dff029b7 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4792: [HUDI-3366] Remove hardcoded logic of disabling metadata table in tests

hudi-bot commented on pull request #4792: URL: https://github.com/apache/hudi/pull/4792#issuecomment-1035981836 ## CI report: * 1b2bdd81dc3f7edea2934b1cd461ee50dff029b7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] codope commented on a change in pull request #3648: [HUDI-2413] fix Sql source's checkpoint issue

codope commented on a change in pull request #3648: URL: https://github.com/apache/hudi/pull/3648#discussion_r804443369 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieDeltaStreamer.java ## @@ -378,6 +378,10 @@ private boolean onDeltaSy

[GitHub] [hudi] pan3793 opened a new issue #4793: [SUPPORT] Integration test broken after upgrade from 0.10.0 to 0.10.1

pan3793 opened a new issue #4793: URL: https://github.com/apache/hudi/issues/4793 **Describe the problem you faced** The Apache Kyuubi (Incubating) Hudi Integration test broken after upgrade from 0.10.0 to 0.10.1. https://github.com/apache/incubator-kyuubi/runs/5152924363

[GitHub] [hudi] boneanxs commented on a change in pull request #4791: [HUDI-3412] TypedProperties no need to create new set when check key …

boneanxs commented on a change in pull request #4791: URL: https://github.com/apache/hudi/pull/4791#discussion_r804452202 ## File path: hudi-common/src/main/java/org/apache/hudi/common/config/TypedProperties.java ## @@ -44,27 +43,22 @@ public TypedProperties(Properties default

[GitHub] [hudi] hudi-bot removed a comment on pull request #4772: [HUDI-3362] Fix restore to rollback pending clustering operations followed by other rolling back other commits

hudi-bot removed a comment on pull request #4772: URL: https://github.com/apache/hudi/pull/4772#issuecomment-1035954025 ## CI report: * d92b221a24d5b04a21e7320e43d04ed699adadc4 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot removed a comment on pull request #4791: [HUDI-3412] TypedProperties no need to create new set when check key …

hudi-bot removed a comment on pull request #4791: URL: https://github.com/apache/hudi/pull/4791#issuecomment-1035958612 ## CI report: * 3c061f10be730b7edb473cdb90be1aa5236a28a7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4772: [HUDI-3362] Fix restore to rollback pending clustering operations followed by other rolling back other commits

hudi-bot commented on pull request #4772: URL: https://github.com/apache/hudi/pull/4772#issuecomment-1035990281 ## CI report: * 9c26457de8ff1bca99f7098a676e17aa1f62b4a8 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #4791: [HUDI-3412] TypedProperties no need to create new set when check key …

hudi-bot commented on pull request #4791: URL: https://github.com/apache/hudi/pull/4791#issuecomment-1035990377 ## CI report: * 3c061f10be730b7edb473cdb90be1aa5236a28a7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] cuibo01 commented on pull request #4699: [HUDI-3336][HUDI-FLINK] Support custom hadoop config options for flink

cuibo01 commented on pull request #4699: URL: https://github.com/apache/hudi/pull/4699#issuecomment-1035995109 > Yes, the `toMap` has similar functionality, but the `getXXX` and `setXXX` methods always prefix the option with prefix, which is not what we want, i still think there is no need

[GitHub] [hudi] hudi-bot removed a comment on pull request #4752: [WIP][HUDI-3088] Use Spark 3.2 as default Spark version

hudi-bot removed a comment on pull request #4752: URL: https://github.com/apache/hudi/pull/4752#issuecomment-1035971918 ## CI report: * bc24e231f7f7e86f0eeac7716f0b1e9293374169 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4752: [WIP][HUDI-3088] Use Spark 3.2 as default Spark version

hudi-bot commented on pull request #4752: URL: https://github.com/apache/hudi/pull/4752#issuecomment-1036004814 ## CI report: * d5f1fbad92cd451d5ac7cf81f5f8612ff18d85ed UNKNOWN * 5dd7309bb81f9b4b5b22de04659d37418a408ba4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot commented on pull request #4775: [WIP] [HUDI-2883] Refactor hive sync tool / config to use reflection and standardize configs

hudi-bot commented on pull request #4775: URL: https://github.com/apache/hudi/pull/4775#issuecomment-1036010826 ## CI report: * 1143c99f4169aa458928a89cb4f6f6dce98d93ae Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4775: [WIP] [HUDI-2883] Refactor hive sync tool / config to use reflection and standardize configs

hudi-bot removed a comment on pull request #4775: URL: https://github.com/apache/hudi/pull/4775#issuecomment-1033699581 ## CI report: * 1143c99f4169aa458928a89cb4f6f6dce98d93ae Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4775: [WIP] [HUDI-2883] Refactor hive sync tool / config to use reflection and standardize configs

hudi-bot commented on pull request #4775: URL: https://github.com/apache/hudi/pull/4775#issuecomment-1036013092 ## CI report: * 1143c99f4169aa458928a89cb4f6f6dce98d93ae Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4775: [WIP] [HUDI-2883] Refactor hive sync tool / config to use reflection and standardize configs

hudi-bot removed a comment on pull request #4775: URL: https://github.com/apache/hudi/pull/4775#issuecomment-1036010826 ## CI report: * 1143c99f4169aa458928a89cb4f6f6dce98d93ae Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4792: [HUDI-3366] Remove hardcoded logic of disabling metadata table in tests

hudi-bot removed a comment on pull request #4792: URL: https://github.com/apache/hudi/pull/4792#issuecomment-1035981836 ## CI report: * 1b2bdd81dc3f7edea2934b1cd461ee50dff029b7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4792: [HUDI-3366] Remove hardcoded logic of disabling metadata table in tests

hudi-bot commented on pull request #4792: URL: https://github.com/apache/hudi/pull/4792#issuecomment-1036035557 ## CI report: * 1b2bdd81dc3f7edea2934b1cd461ee50dff029b7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[jira] [Created] (HUDI-3413) fix jackson parse error when empty message from JsonKafkaSource Using HoodieDeltaStreamer

xiang zhang created HUDI-3413: - Summary: fix jackson parse error when empty message from JsonKafkaSource Using HoodieDeltaStreamer Key: HUDI-3413 URL: https://issues.apache.org/jira/browse/HUDI-3413 Pro

[jira] [Updated] (HUDI-3413) fix jackson parse error when empty message from JsonKafkaSource Using HoodieDeltaStreamer

[ https://issues.apache.org/jira/browse/HUDI-3413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xiang zhang updated HUDI-3413: -- Priority: Minor (was: Major) > fix jackson parse error when empty message from JsonKafkaSource Using

[GitHub] [hudi] zhangxiang17 opened a new pull request #4794: [HUDI-3413]fix jackson parse error when empty message from JsonKafkaS…

zhangxiang17 opened a new pull request #4794: URL: https://github.com/apache/hudi/pull/4794 …ource Using HoodieDeltaStreamer ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a

[jira] [Updated] (HUDI-3413) fix jackson parse error when empty message from JsonKafkaSource Using HoodieDeltaStreamer

[ https://issues.apache.org/jira/browse/HUDI-3413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3413: - Labels: pull-request-available (was: ) > fix jackson parse error when empty message from JsonKaf

[GitHub] [hudi] hudi-bot commented on pull request #4794: [HUDI-3413]fix jackson parse error when empty message from JsonKafkaS…

hudi-bot commented on pull request #4794: URL: https://github.com/apache/hudi/pull/4794#issuecomment-1036046786 ## CI report: * d313daba428b2b0007ee688219a6273ddaa7aea0 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot removed a comment on pull request #4794: [HUDI-3413] Fix empty message jackson parse error in JsonKafkaSource

hudi-bot removed a comment on pull request #4794: URL: https://github.com/apache/hudi/pull/4794#issuecomment-1036046786 ## CI report: * d313daba428b2b0007ee688219a6273ddaa7aea0 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4794: [HUDI-3413] Fix empty message jackson parse error in JsonKafkaSource

hudi-bot commented on pull request #4794: URL: https://github.com/apache/hudi/pull/4794#issuecomment-1036048922 ## CI report: * d313daba428b2b0007ee688219a6273ddaa7aea0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] wangxianghu commented on pull request #4794: [HUDI-3413] Fix empty message jackson parse error in JsonKafkaSource

wangxianghu commented on pull request #4794: URL: https://github.com/apache/hudi/pull/4794#issuecomment-1036048925 @zhangxiang17 Thanks for this fix :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hudi] hudi-bot removed a comment on pull request #4753: [HUDI-3370] The files recorded in the commit may not match the actual ones for MOR Compaction

hudi-bot removed a comment on pull request #4753: URL: https://github.com/apache/hudi/pull/4753#issuecomment-1035874974 ## CI report: * ea61391876e0f18cd1cde28dcbc1de4623ab66de Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4775: [WIP] [HUDI-2883] Refactor hive sync tool / config to use reflection and standardize configs

hudi-bot removed a comment on pull request #4775: URL: https://github.com/apache/hudi/pull/4775#issuecomment-1036013092 ## CI report: * 1143c99f4169aa458928a89cb4f6f6dce98d93ae Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4753: [HUDI-3370] The files recorded in the commit may not match the actual ones for MOR Compaction

hudi-bot commented on pull request #4753: URL: https://github.com/apache/hudi/pull/4753#issuecomment-1036055497 ## CI report: * ea61391876e0f18cd1cde28dcbc1de4623ab66de Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #4775: [WIP] [HUDI-2883] Refactor hive sync tool / config to use reflection and standardize configs

hudi-bot commented on pull request #4775: URL: https://github.com/apache/hudi/pull/4775#issuecomment-1036055589 ## CI report: * 93db7c550338eb4368a69d3a5ee0c33c68a5b5cd Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4753: [HUDI-3370] The files recorded in the commit may not match the actual ones for MOR Compaction

hudi-bot removed a comment on pull request #4753: URL: https://github.com/apache/hudi/pull/4753#issuecomment-1036055497 ## CI report: * ea61391876e0f18cd1cde28dcbc1de4623ab66de Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4753: [HUDI-3370] The files recorded in the commit may not match the actual ones for MOR Compaction

hudi-bot commented on pull request #4753: URL: https://github.com/apache/hudi/pull/4753#issuecomment-1036057820 ## CI report: * ea61391876e0f18cd1cde28dcbc1de4623ab66de Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] geoffroyatkwiff commented on issue #4778: [SUPPORT] Row with _hoodie_is_deleted=True stored into target table

geoffroyatkwiff commented on issue #4778: URL: https://github.com/apache/hudi/issues/4778#issuecomment-1036073470 Hey @nsivabalan , I'm using `upsert`. I'm using a timestamp for `hoodie.datasource.write.precombine.field`. So, in the case of a dataframe being processed that would contain

[GitHub] [hudi] hudi-bot removed a comment on pull request #4794: [HUDI-3413] Fix empty message jackson parse error in JsonKafkaSource

hudi-bot removed a comment on pull request #4794: URL: https://github.com/apache/hudi/pull/4794#issuecomment-1036048922 ## CI report: * d313daba428b2b0007ee688219a6273ddaa7aea0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4794: [HUDI-3413] Fix empty message jackson parse error in JsonKafkaSource

hudi-bot commented on pull request #4794: URL: https://github.com/apache/hudi/pull/4794#issuecomment-1036095788 ## CI report: * d313daba428b2b0007ee688219a6273ddaa7aea0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] andykrk commented on issue #4604: [SUPPORT] Archive functionality fails

andykrk commented on issue #4604: URL: https://github.com/apache/hudi/issues/4604#issuecomment-1036096714 Hello, I will try to describe the procedure with more details. It's very close to what you have described though. Please hold on with reproduction attempt and let me try again n

[GitHub] [hudi] hudi-bot commented on pull request #4752: [WIP][HUDI-3088] Use Spark 3.2 as default Spark version

hudi-bot commented on pull request #4752: URL: https://github.com/apache/hudi/pull/4752#issuecomment-1036108147 ## CI report: * d5f1fbad92cd451d5ac7cf81f5f8612ff18d85ed UNKNOWN * 5dd7309bb81f9b4b5b22de04659d37418a408ba4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4752: [WIP][HUDI-3088] Use Spark 3.2 as default Spark version

hudi-bot removed a comment on pull request #4752: URL: https://github.com/apache/hudi/pull/4752#issuecomment-1036004814 ## CI report: * d5f1fbad92cd451d5ac7cf81f5f8612ff18d85ed UNKNOWN * 5dd7309bb81f9b4b5b22de04659d37418a408ba4 Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot removed a comment on pull request #4753: [HUDI-3370] The files recorded in the commit may not match the actual ones for MOR Compaction

hudi-bot removed a comment on pull request #4753: URL: https://github.com/apache/hudi/pull/4753#issuecomment-1036057820 ## CI report: * ea61391876e0f18cd1cde28dcbc1de4623ab66de Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4753: [HUDI-3370] The files recorded in the commit may not match the actual ones for MOR Compaction

hudi-bot commented on pull request #4753: URL: https://github.com/apache/hudi/pull/4753#issuecomment-1036112386 ## CI report: * 5834378fe9ef4ee4ad4e44df6e0a61362e522fe5 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] zhangyue19921010 commented on pull request #4753: [HUDI-3370] The files recorded in the commit may not match the actual ones for MOR Compaction

zhangyue19921010 commented on pull request #4753: URL: https://github.com/apache/hudi/pull/4753#issuecomment-1036129050 > Thanks for the contribution @zhangyue19921010 , overall looks good, i left some minor comments. Thanks @danny0405 All comments are addressed. Also CI passed. PTAL

[GitHub] [hudi] zhangyue19921010 commented on a change in pull request #4753: [HUDI-3370] The files recorded in the commit may not match the actual ones for MOR Compaction

zhangyue19921010 commented on a change in pull request #4753: URL: https://github.com/apache/hudi/pull/4753#discussion_r804578059 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/HoodieSparkCompactor.java ## @@ -43,19 +48,24 @@ public HoodieSpar

[GitHub] [hudi] zhangyue19921010 commented on a change in pull request #4753: [HUDI-3370] The files recorded in the commit may not match the actual ones for MOR Compaction

zhangyue19921010 commented on a change in pull request #4753: URL: https://github.com/apache/hudi/pull/4753#discussion_r804578214 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieCompactor.java ## @@ -293,4 +297,17 @@ private String getSchemaFromLates

[GitHub] [hudi] XuQianJin-Stars commented on pull request #4752: [WIP][HUDI-3088] Use Spark 3.2 as default Spark version

XuQianJin-Stars commented on pull request #4752: URL: https://github.com/apache/hudi/pull/4752#issuecomment-1036154342 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [hudi] hudi-bot removed a comment on pull request #4752: [WIP][HUDI-3088] Use Spark 3.2 as default Spark version

hudi-bot removed a comment on pull request #4752: URL: https://github.com/apache/hudi/pull/4752#issuecomment-1036108147 ## CI report: * d5f1fbad92cd451d5ac7cf81f5f8612ff18d85ed UNKNOWN * 5dd7309bb81f9b4b5b22de04659d37418a408ba4 Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot commented on pull request #4752: [WIP][HUDI-3088] Use Spark 3.2 as default Spark version

hudi-bot commented on pull request #4752: URL: https://github.com/apache/hudi/pull/4752#issuecomment-1036155469 ## CI report: * d5f1fbad92cd451d5ac7cf81f5f8612ff18d85ed UNKNOWN * 5dd7309bb81f9b4b5b22de04659d37418a408ba4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] XuQianJin-Stars removed a comment on pull request #4752: [WIP][HUDI-3088] Use Spark 3.2 as default Spark version

XuQianJin-Stars removed a comment on pull request #4752: URL: https://github.com/apache/hudi/pull/4752#issuecomment-1036154342 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] hudi-bot removed a comment on pull request #4752: [WIP][HUDI-3088] Use Spark 3.2 as default Spark version

hudi-bot removed a comment on pull request #4752: URL: https://github.com/apache/hudi/pull/4752#issuecomment-1036155469 ## CI report: * d5f1fbad92cd451d5ac7cf81f5f8612ff18d85ed UNKNOWN * 5dd7309bb81f9b4b5b22de04659d37418a408ba4 Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot commented on pull request #4752: [WIP][HUDI-3088] Use Spark 3.2 as default Spark version

hudi-bot commented on pull request #4752: URL: https://github.com/apache/hudi/pull/4752#issuecomment-1036164056 ## CI report: * d5f1fbad92cd451d5ac7cf81f5f8612ff18d85ed UNKNOWN * 5dd7309bb81f9b4b5b22de04659d37418a408ba4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4752: [WIP][HUDI-3088] Use Spark 3.2 as default Spark version

hudi-bot removed a comment on pull request #4752: URL: https://github.com/apache/hudi/pull/4752#issuecomment-1036164056 ## CI report: * d5f1fbad92cd451d5ac7cf81f5f8612ff18d85ed UNKNOWN * 5dd7309bb81f9b4b5b22de04659d37418a408ba4 Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot commented on pull request #4752: [WIP][HUDI-3088] Use Spark 3.2 as default Spark version

hudi-bot commented on pull request #4752: URL: https://github.com/apache/hudi/pull/4752#issuecomment-1036187779 ## CI report: * d5f1fbad92cd451d5ac7cf81f5f8612ff18d85ed UNKNOWN * 5dd7309bb81f9b4b5b22de04659d37418a408ba4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4752: [WIP][HUDI-3088] Use Spark 3.2 as default Spark version

hudi-bot removed a comment on pull request #4752: URL: https://github.com/apache/hudi/pull/4752#issuecomment-1036187779 ## CI report: * d5f1fbad92cd451d5ac7cf81f5f8612ff18d85ed UNKNOWN * 5dd7309bb81f9b4b5b22de04659d37418a408ba4 Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot commented on pull request #4752: [WIP][HUDI-3088] Use Spark 3.2 as default Spark version

hudi-bot commented on pull request #4752: URL: https://github.com/apache/hudi/pull/4752#issuecomment-1036198959 ## CI report: * d5f1fbad92cd451d5ac7cf81f5f8612ff18d85ed UNKNOWN * 42cbe5434d4b24acd24d3dca5f1ff027f5e95fcb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot commented on pull request #4761: [HUDI-3356][HUDI-3142][HUDI-1492] Metadata column stats index - handling delta writes

hudi-bot commented on pull request #4761: URL: https://github.com/apache/hudi/pull/4761#issuecomment-1036203793 ## CI report: * 974c5ecb0234bcf34b0f0ef86057decb9812 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4761: [HUDI-3356][HUDI-3142][HUDI-1492] Metadata column stats index - handling delta writes

hudi-bot removed a comment on pull request #4761: URL: https://github.com/apache/hudi/pull/4761#issuecomment-1033224445 ## CI report: * 974c5ecb0234bcf34b0f0ef86057decb9812 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4761: [HUDI-3356][HUDI-3142][HUDI-1492] Metadata column stats index - handling delta writes

hudi-bot removed a comment on pull request #4761: URL: https://github.com/apache/hudi/pull/4761#issuecomment-1036203793 ## CI report: * 974c5ecb0234bcf34b0f0ef86057decb9812 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4761: [HUDI-3356][HUDI-3142][HUDI-1492] Metadata column stats index - handling delta writes

hudi-bot commented on pull request #4761: URL: https://github.com/apache/hudi/pull/4761#issuecomment-1036206376 ## CI report: * 974c5ecb0234bcf34b0f0ef86057decb9812 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #4795: [HUDI-1576] Make archiving an async service

hudi-bot commented on pull request #4795: URL: https://github.com/apache/hudi/pull/4795#issuecomment-1036211540 ## CI report: * 361f176bbe028130f1e5a23d7673c22f169ec7b7 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[jira] [Updated] (HUDI-1576) Add ability to perform archival synchronously

[ https://issues.apache.org/jira/browse/HUDI-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1576: - Labels: pull-request-available (was: ) > Add ability to perform archival synchronously >

[GitHub] [hudi] hudi-bot removed a comment on pull request #4795: [HUDI-1576] Make archiving an async service

hudi-bot removed a comment on pull request #4795: URL: https://github.com/apache/hudi/pull/4795#issuecomment-1036211540 ## CI report: * 361f176bbe028130f1e5a23d7673c22f169ec7b7 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4795: [HUDI-1576] Make archiving an async service

hudi-bot commented on pull request #4795: URL: https://github.com/apache/hudi/pull/4795#issuecomment-1036214116 ## CI report: * 361f176bbe028130f1e5a23d7673c22f169ec7b7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4795: [HUDI-1576] Make archiving an async service

hudi-bot removed a comment on pull request #4795: URL: https://github.com/apache/hudi/pull/4795#issuecomment-1036214116 ## CI report: * 361f176bbe028130f1e5a23d7673c22f169ec7b7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4795: [HUDI-1576] Make archiving an async service

hudi-bot commented on pull request #4795: URL: https://github.com/apache/hudi/pull/4795#issuecomment-1036219570 ## CI report: * 361f176bbe028130f1e5a23d7673c22f169ec7b7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4795: [HUDI-1576] Make archiving an async service

hudi-bot removed a comment on pull request #4795: URL: https://github.com/apache/hudi/pull/4795#issuecomment-1036219570 ## CI report: * 361f176bbe028130f1e5a23d7673c22f169ec7b7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4795: [HUDI-1576] Make archiving an async service

hudi-bot commented on pull request #4795: URL: https://github.com/apache/hudi/pull/4795#issuecomment-1036222166 ## CI report: * 361f176bbe028130f1e5a23d7673c22f169ec7b7 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #4795: [HUDI-1576] Make archiving an async service

hudi-bot removed a comment on pull request #4795: URL: https://github.com/apache/hudi/pull/4795#issuecomment-1036222166 ## CI report: * 361f176bbe028130f1e5a23d7673c22f169ec7b7 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4795: [HUDI-1576] Make archiving an async service

hudi-bot commented on pull request #4795: URL: https://github.com/apache/hudi/pull/4795#issuecomment-1036249771 ## CI report: * 361f176bbe028130f1e5a23d7673c22f169ec7b7 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #4761: [HUDI-3356][HUDI-3142][HUDI-1492] Metadata column stats index - handling delta writes

hudi-bot removed a comment on pull request #4761: URL: https://github.com/apache/hudi/pull/4761#issuecomment-1036206376 ## CI report: * 974c5ecb0234bcf34b0f0ef86057decb9812 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4761: [HUDI-3356][HUDI-3142][HUDI-1492] Metadata column stats index - handling delta writes

hudi-bot commented on pull request #4761: URL: https://github.com/apache/hudi/pull/4761#issuecomment-1036260406 ## CI report: * 6d40a75e6d159f9c961f2e960760022af915beee Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4795: [HUDI-1576] Make archiving an async service

hudi-bot removed a comment on pull request #4795: URL: https://github.com/apache/hudi/pull/4795#issuecomment-1036249771 ## CI report: * 361f176bbe028130f1e5a23d7673c22f169ec7b7 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4795: [HUDI-1576] Make archiving an async service

hudi-bot commented on pull request #4795: URL: https://github.com/apache/hudi/pull/4795#issuecomment-1036268815 ## CI report: * 361f176bbe028130f1e5a23d7673c22f169ec7b7 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #4795: [HUDI-1576] Make archiving an async service

hudi-bot removed a comment on pull request #4795: URL: https://github.com/apache/hudi/pull/4795#issuecomment-1036268815 ## CI report: * 361f176bbe028130f1e5a23d7673c22f169ec7b7 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4795: [HUDI-1576] Make archiving an async service

hudi-bot commented on pull request #4795: URL: https://github.com/apache/hudi/pull/4795#issuecomment-1036271576 ## CI report: * 361f176bbe028130f1e5a23d7673c22f169ec7b7 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #4795: [HUDI-1576] Make archiving an async service

hudi-bot removed a comment on pull request #4795: URL: https://github.com/apache/hudi/pull/4795#issuecomment-1036271576 ## CI report: * 361f176bbe028130f1e5a23d7673c22f169ec7b7 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4795: [HUDI-1576] Make archiving an async service

hudi-bot commented on pull request #4795: URL: https://github.com/apache/hudi/pull/4795#issuecomment-1036274270 ## CI report: * 068629b671abbf6fa2f634657d82cf6363eb5f19 UNKNOWN * 9fbfabc1b190d493e54fda5edced1baa18175c02 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-or

[jira] [Commented] (HUDI-3391) presto and hive beeline fails to read MOR table w/ 2 or more array fields

[ https://issues.apache.org/jira/browse/HUDI-3391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17490960#comment-17490960 ] Harsha Gudladona commented on HUDI-3391: Adding some more context on the dependenc

[jira] [Comment Edited] (HUDI-3391) presto and hive beeline fails to read MOR table w/ 2 or more array fields

[ https://issues.apache.org/jira/browse/HUDI-3391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17490960#comment-17490960 ] Harsha Gudladona edited comment on HUDI-3391 at 2/11/22, 2:47 PM: --

[GitHub] [hudi] hudi-bot commented on pull request #4795: [HUDI-1576] Make archiving an async service

hudi-bot commented on pull request #4795: URL: https://github.com/apache/hudi/pull/4795#issuecomment-1036289601 ## CI report: * 068629b671abbf6fa2f634657d82cf6363eb5f19 UNKNOWN * 9fbfabc1b190d493e54fda5edced1baa18175c02 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-or

[GitHub] [hudi] hudi-bot removed a comment on pull request #4795: [HUDI-1576] Make archiving an async service

hudi-bot removed a comment on pull request #4795: URL: https://github.com/apache/hudi/pull/4795#issuecomment-1036274270 ## CI report: * 068629b671abbf6fa2f634657d82cf6363eb5f19 UNKNOWN * 9fbfabc1b190d493e54fda5edced1baa18175c02 Azure: [CANCELED](https://dev.azure.com/apache-hu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4761: [HUDI-3356][HUDI-3142][HUDI-1492] Metadata column stats index - handling delta writes

hudi-bot removed a comment on pull request #4761: URL: https://github.com/apache/hudi/pull/4761#issuecomment-1036260406 ## CI report: * 6d40a75e6d159f9c961f2e960760022af915beee Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4761: [HUDI-3356][HUDI-3142][HUDI-1492] Metadata column stats index - handling delta writes

hudi-bot commented on pull request #4761: URL: https://github.com/apache/hudi/pull/4761#issuecomment-1036293797 ## CI report: * 6d40a75e6d159f9c961f2e960760022af915beee Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4761: [HUDI-3356][HUDI-3142][HUDI-1492] Metadata column stats index - handling delta writes

hudi-bot removed a comment on pull request #4761: URL: https://github.com/apache/hudi/pull/4761#issuecomment-1036293797 ## CI report: * 6d40a75e6d159f9c961f2e960760022af915beee Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4761: [HUDI-3356][HUDI-3142][HUDI-1492] Metadata column stats index - handling delta writes

hudi-bot commented on pull request #4761: URL: https://github.com/apache/hudi/pull/4761#issuecomment-1036297912 ## CI report: * 6d40a75e6d159f9c961f2e960760022af915beee Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] nsivabalan commented on issue #4788: [SUPPORT] run_sync_tool throws java.lang.NoSuchFieldError: TIMESTAMP_MICROS

nsivabalan commented on issue #4788: URL: https://github.com/apache/hudi/issues/4788#issuecomment-1036301189 likely its a jar version mismatch. class of interest if from parquet-avro. Do you happened to have parquet-avro in your class path. if yes, can you remove it and give it a try.

[GitHub] [hudi] nsivabalan commented on issue #4778: [SUPPORT] Row with _hoodie_is_deleted=True stored into target table

nsivabalan commented on issue #4778: URL: https://github.com/apache/hudi/issues/4778#issuecomment-1036303695 sorry. is your requirement, if we have insert and delete for the same record within one batch thats being ingested, you prefer final snapshot in hudi to show the insert record and n

[GitHub] [hudi] nsivabalan commented on issue #4778: [SUPPORT] Row with _hoodie_is_deleted=True stored into target table

nsivabalan commented on issue #4778: URL: https://github.com/apache/hudi/issues/4778#issuecomment-1036305248 guess during dedup, hudi goes by preCombine field value. whichever record has higher preCombine value will win. -- This is an automated message from the Apache Git Service. To re

[GitHub] [hudi] geoffroyatkwiff commented on issue #4778: [SUPPORT] Row with _hoodie_is_deleted=True stored into target table

geoffroyatkwiff commented on issue #4778: URL: https://github.com/apache/hudi/issues/4778#issuecomment-1036325464 > sorry. is your requirement, if we have insert and delete for the same record within one batch thats being ingested, you prefer final snapshot in hudi to show the insert recor

[GitHub] [hudi] nsivabalan commented on issue #4784: [SUPPORT] Partition column not appearing in spark dataframe

nsivabalan commented on issue #4784: URL: https://github.com/apache/hudi/issues/4784#issuecomment-1036330505 I could not reproduce the partitioning issue you are facing. local spark shell ``` import java.sql.Timestamp import spark.implicits._ import org.apache.hudi

[GitHub] [hudi] nsivabalan edited a comment on issue #4784: [SUPPORT] Partition column not appearing in spark dataframe

nsivabalan edited a comment on issue #4784: URL: https://github.com/apache/hudi/issues/4784#issuecomment-1036330505 I could not reproduce the partitioning issue you are facing. I could see my partition is well formed and I could see the two original columns which i used to generate the par

  1   2   3   4   >