[GitHub] [hudi] boneanxs commented on pull request #5027: [HUDI-3612] Should create new TypedProperties when modifying it

boneanxs commented on pull request #5027: URL: https://github.com/apache/hudi/pull/5027#issuecomment-1079627877 @codope can merge this now -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [hudi] wqwl611 opened a new issue #5136: [SUPPORT] Empty parquet "is not a Parquet file (length is too low: 0)"

wqwl611 opened a new issue #5136: URL: https://github.com/apache/hudi/issues/5136 sometimes i will find this bug, and i delete the empty parquet file, it will work but I guess I lost data. Steps to reproduce the behavior: i don't know how to reproduce commit info show totalWr

[GitHub] [hudi] hudi-bot commented on pull request #4945: [HUDI-3538] Support Compaction Command Based on Call Procedure Command for Spark SQL

hudi-bot commented on pull request #4945: URL: https://github.com/apache/hudi/pull/4945#issuecomment-1079630438 ## CI report: * f56cb371849890e9ceb200861b0c4eca598983f9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4945: [HUDI-3538] Support Compaction Command Based on Call Procedure Command for Spark SQL

hudi-bot removed a comment on pull request #4945: URL: https://github.com/apache/hudi/pull/4945#issuecomment-1079597091 ## CI report: * 1e6045ac77f45a33e66fa1c5be6027cb4fd94a52 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

hudi-bot commented on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-107968 ## CI report: * 9ba5c351c32f9f364c30b4bc9a814075150d9728 UNKNOWN * 53b4cec7d05dd0bb5a622ffb33597471da7711de UNKNOWN * e3140bb9abef7d5e9dc0405b9f8b12f11c2438da UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

hudi-bot removed a comment on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-1079604726 ## CI report: * 9ba5c351c32f9f364c30b4bc9a814075150d9728 UNKNOWN * 53b4cec7d05dd0bb5a622ffb33597471da7711de UNKNOWN * e3140bb9abef7d5e9dc0405b9f8b12f11c243

[GitHub] [hudi] nsivabalan commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

nsivabalan commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835722655 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java ## @@ -915,6 +917,39 @@ public boolean sched

[GitHub] [hudi] peanut-chenzhong removed a comment on pull request #5042: Three bulk_insert files are concurrently submitted and executed with a difference of 2s, the insert fails occasionally.

peanut-chenzhong removed a comment on pull request #5042: URL: https://github.com/apache/hudi/pull/5042#issuecomment-1068771256 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [hudi] peanut-chenzhong commented on pull request #5042: Three bulk_insert files are concurrently submitted and executed with a difference of 2s, the insert fails occasionally.

peanut-chenzhong commented on pull request #5042: URL: https://github.com/apache/hudi/pull/5042#issuecomment-1079633909 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [hudi] hudi-bot removed a comment on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

hudi-bot removed a comment on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-107968 ## CI report: * 9ba5c351c32f9f364c30b4bc9a814075150d9728 UNKNOWN * 53b4cec7d05dd0bb5a622ffb33597471da7711de UNKNOWN * e3140bb9abef7d5e9dc0405b9f8b12f11c243

[GitHub] [hudi] hudi-bot commented on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

hudi-bot commented on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-1079634246 ## CI report: * 9ba5c351c32f9f364c30b4bc9a814075150d9728 UNKNOWN * 53b4cec7d05dd0bb5a622ffb33597471da7711de UNKNOWN * e3140bb9abef7d5e9dc0405b9f8b12f11c2438da UNKN

[GitHub] [hudi] hudi-bot commented on pull request #5042: Three bulk_insert files are concurrently submitted and executed with a difference of 2s, the insert fails occasionally.

hudi-bot commented on pull request #5042: URL: https://github.com/apache/hudi/pull/5042#issuecomment-1079634270 ## CI report: * 7ee24be4d11864af37bf300250d571e15d5f9ae9 UNKNOWN * Unknown: [CANCELED](TBD) * 9a9e544ba48a52c7b54134fc9533c3e5a51ccfff UNKNOWN Bot com

[GitHub] [hudi] hudi-bot removed a comment on pull request #5042: Three bulk_insert files are concurrently submitted and executed with a difference of 2s, the insert fails occasionally.

hudi-bot removed a comment on pull request #5042: URL: https://github.com/apache/hudi/pull/5042#issuecomment-1068826183 ## CI report: * 7ee24be4d11864af37bf300250d571e15d5f9ae9 UNKNOWN * 107dab60f6541702644f4c87b7e92ba68f2fe6ef Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot removed a comment on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

hudi-bot removed a comment on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1079624873 ## CI report: * 52be34d7d5e025180415c46e64a3e2145c29e498 UNKNOWN * 78e86dd1953cc4d6bf10ca808a7bcffe22b4b587 UNKNOWN * fa9cee18b16f1b11ed039a9da3c490f017775

[GitHub] [hudi] hudi-bot commented on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

hudi-bot commented on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1079637508 ## CI report: * 52be34d7d5e025180415c46e64a3e2145c29e498 UNKNOWN * 78e86dd1953cc4d6bf10ca808a7bcffe22b4b587 UNKNOWN * fa9cee18b16f1b11ed039a9da3c490f017775e8d UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4945: [HUDI-3538] Support Compaction Command Based on Call Procedure Command for Spark SQL

hudi-bot removed a comment on pull request #4945: URL: https://github.com/apache/hudi/pull/4945#issuecomment-1079630438 ## CI report: * f56cb371849890e9ceb200861b0c4eca598983f9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4945: [HUDI-3538] Support Compaction Command Based on Call Procedure Command for Spark SQL

hudi-bot commented on pull request #4945: URL: https://github.com/apache/hudi/pull/4945#issuecomment-1079638759 ## CI report: * f56cb371849890e9ceb200861b0c4eca598983f9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #4945: [HUDI-3538] Support Compaction Command Based on Call Procedure Command for Spark SQL

hudi-bot commented on pull request #4945: URL: https://github.com/apache/hudi/pull/4945#issuecomment-1079639187 ## CI report: * f56cb371849890e9ceb200861b0c4eca598983f9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4945: [HUDI-3538] Support Compaction Command Based on Call Procedure Command for Spark SQL

hudi-bot removed a comment on pull request #4945: URL: https://github.com/apache/hudi/pull/4945#issuecomment-1079638759 ## CI report: * f56cb371849890e9ceb200861b0c4eca598983f9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

hudi-bot commented on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-1079644731 ## CI report: * 9ba5c351c32f9f364c30b4bc9a814075150d9728 UNKNOWN * 53b4cec7d05dd0bb5a622ffb33597471da7711de UNKNOWN * e3140bb9abef7d5e9dc0405b9f8b12f11c2438da UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

hudi-bot removed a comment on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-1079634246 ## CI report: * 9ba5c351c32f9f364c30b4bc9a814075150d9728 UNKNOWN * 53b4cec7d05dd0bb5a622ffb33597471da7711de UNKNOWN * e3140bb9abef7d5e9dc0405b9f8b12f11c243

[GitHub] [hudi] xiarixiaoyao commented on issue #5107: [SUPPORT] High performance costs of AvroSerializer in Datasource writing

xiarixiaoyao commented on issue #5107: URL: https://github.com/apache/hudi/issues/5107#issuecomment-1079650016 @YuweiXiao Thank you for discussing this issue， have you test on newest master branch. I found the situation very even worse -- This is an automated message from the Apache

[GitHub] [hudi] hudi-bot commented on pull request #4945: [HUDI-3538] Support Compaction Command Based on Call Procedure Command for Spark SQL

hudi-bot commented on pull request #4945: URL: https://github.com/apache/hudi/pull/4945#issuecomment-1079650411 ## CI report: * ce55ea05eb9d27917fce72a3e40678ee8a5ed55e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4945: [HUDI-3538] Support Compaction Command Based on Call Procedure Command for Spark SQL

hudi-bot removed a comment on pull request #4945: URL: https://github.com/apache/hudi/pull/4945#issuecomment-1079639187 ## CI report: * f56cb371849890e9ceb200861b0c4eca598983f9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] codope commented on pull request #5027: [HUDI-3612] Should create new TypedProperties when modifying it

codope commented on pull request #5027: URL: https://github.com/apache/hudi/pull/5027#issuecomment-107965 Thanks @boneanxs for working on this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [hudi] codope merged pull request #5027: [HUDI-3612] Should create new TypedProperties when modifying it

codope merged pull request #5027: URL: https://github.com/apache/hudi/pull/5027 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr..

[hudi] branch master updated (0c09a97 -> 57b4f39)

This is an automated email from the ASF dual-hosted git repository. codope pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 0c09a97 [HUDI-3435] Do not throw exception when instant to rollback does not exist in metadata table active timeline

[jira] [Closed] (HUDI-3612) Clustering strategy should create new TypedProperties when modifying it

[ https://issues.apache.org/jira/browse/HUDI-3612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-3612. - Fix Version/s: 0.11.0 Resolution: Fixed > Clustering strategy should create new TypedProperties whe

[GitHub] [hudi] hudi-bot removed a comment on pull request #5076: [HUDI-2757] Implement Hudi AWS Glue sync

hudi-bot removed a comment on pull request #5076: URL: https://github.com/apache/hudi/pull/5076#issuecomment-1074283748 ## CI report: * 7bb94c049a0c5800d54146a5b87878c0fc7d7e60 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #5076: [HUDI-2757] Implement Hudi AWS Glue sync

hudi-bot commented on pull request #5076: URL: https://github.com/apache/hudi/pull/5076#issuecomment-1079662774 ## CI report: * 7bb94c049a0c5800d54146a5b87878c0fc7d7e60 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #5076: [HUDI-2757] Implement Hudi AWS Glue sync

hudi-bot removed a comment on pull request #5076: URL: https://github.com/apache/hudi/pull/5076#issuecomment-1079662774 ## CI report: * 7bb94c049a0c5800d54146a5b87878c0fc7d7e60 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #5076: [HUDI-2757] Implement Hudi AWS Glue sync

hudi-bot commented on pull request #5076: URL: https://github.com/apache/hudi/pull/5076#issuecomment-1079663243 ## CI report: * 7bb94c049a0c5800d54146a5b87878c0fc7d7e60 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] xushiyan commented on a change in pull request #5076: [HUDI-2757] Implement Hudi AWS Glue sync

xushiyan commented on a change in pull request #5076: URL: https://github.com/apache/hudi/pull/5076#discussion_r835752737 ## File path: hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/AbstractHiveSyncHoodieClient.java ## @@ -0,0 +1,130 @@ +/* + * Licensed to the Ap

[GitHub] [hudi] hudi-bot commented on pull request #5076: [HUDI-2757] Implement Hudi AWS Glue sync

hudi-bot commented on pull request #5076: URL: https://github.com/apache/hudi/pull/5076#issuecomment-1079664518 ## CI report: * 5b94ceae616194a85bd4014e77f9c007766a3b08 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #5076: [HUDI-2757] Implement Hudi AWS Glue sync

hudi-bot removed a comment on pull request #5076: URL: https://github.com/apache/hudi/pull/5076#issuecomment-1079663243 ## CI report: * 7bb94c049a0c5800d54146a5b87878c0fc7d7e60 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] xushiyan commented on a change in pull request #5076: [HUDI-2757] Implement Hudi AWS Glue sync

xushiyan commented on a change in pull request #5076: URL: https://github.com/apache/hudi/pull/5076#discussion_r835752983 ## File path: hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/AbstractHiveSyncHoodieClient.java ## @@ -0,0 +1,130 @@ +/* + * Licensed to the Ap

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835753092 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java ## @@ -915,6 +917,39 @@ public boolean scheduleC

[GitHub] [hudi] xushiyan commented on a change in pull request #5076: [HUDI-2757] Implement Hudi AWS Glue sync

xushiyan commented on a change in pull request #5076: URL: https://github.com/apache/hudi/pull/5076#discussion_r835753348 ## File path: hudi-aws/src/main/java/org/apache/hudi/aws/sync/AWSGlueCatalogClient.java ## @@ -0,0 +1,474 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835754540 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java ## @@ -621,8 +635,14 @@ private void

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835755588 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/bloom/HoodieBloomIndex.java ## @@ -120,7 +120,7 @@ public HoodieBloomIndex(H

[GitHub] [hudi] hudi-bot removed a comment on pull request #5076: [HUDI-2757] Implement Hudi AWS Glue sync

hudi-bot removed a comment on pull request #5076: URL: https://github.com/apache/hudi/pull/5076#issuecomment-1079664518 ## CI report: * 5b94ceae616194a85bd4014e77f9c007766a3b08 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #5076: [HUDI-2757] Implement Hudi AWS Glue sync

hudi-bot commented on pull request #5076: URL: https://github.com/apache/hudi/pull/5076#issuecomment-1079670938 ## CI report: * 5b94ceae616194a85bd4014e77f9c007766a3b08 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #5076: [HUDI-2757] Implement Hudi AWS Glue sync

hudi-bot removed a comment on pull request #5076: URL: https://github.com/apache/hudi/pull/5076#issuecomment-1079670938 ## CI report: * 5b94ceae616194a85bd4014e77f9c007766a3b08 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #5076: [HUDI-2757] Implement Hudi AWS Glue sync

hudi-bot commented on pull request #5076: URL: https://github.com/apache/hudi/pull/5076#issuecomment-1079672614 ## CI report: * 5b94ceae616194a85bd4014e77f9c007766a3b08 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835757648 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java ## @@ -608,7 +624,7 @@ private void

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835758046 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java ## @@ -620,8 +636,14 @@ private void

[jira] [Created] (HUDI-3718) Support concurrent writes while dropping index

Sagar Sumit created HUDI-3718: - Summary: Support concurrent writes while dropping index Key: HUDI-3718 URL: https://issues.apache.org/jira/browse/HUDI-3718 Project: Apache Hudi Issue Type: New Fe

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835762079 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java ## @@ -645,12 +669,36 @@ private voi

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835762079 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java ## @@ -645,12 +669,36 @@ private voi

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835763190 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java ## @@ -645,12 +669,36 @@ private voi

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835763435 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java ## @@ -663,20 +711,82 @@ private Met

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835763700 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java ## @@ -663,20 +711,82 @@ private Met

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835764267 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/index/RunIndexActionExecutor.java ## @@ -0,0 +1,189 @@ +/* + * Licens

[GitHub] [hudi] hudi-bot commented on pull request #5076: [HUDI-2757] Implement Hudi AWS Glue sync

hudi-bot commented on pull request #5076: URL: https://github.com/apache/hudi/pull/5076#issuecomment-1079693233 ## CI report: * a1159174a7ccfe1705d03807650406e635eff594 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #5076: [HUDI-2757] Implement Hudi AWS Glue sync

hudi-bot removed a comment on pull request #5076: URL: https://github.com/apache/hudi/pull/5076#issuecomment-1079672614 ## CI report: * 5b94ceae616194a85bd4014e77f9c007766a3b08 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835764478 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/index/RunIndexActionExecutor.java ## @@ -0,0 +1,189 @@ +/* + * Licens

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835766943 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/index/RunIndexActionExecutor.java ## @@ -0,0 +1,189 @@ +/* + * Licens

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835767080 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/index/RunIndexActionExecutor.java ## @@ -0,0 +1,189 @@ +/* + * Licens

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835767254 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/index/RunIndexActionExecutor.java ## @@ -0,0 +1,263 @@ +/* + * Licens

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835767497 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/index/RunIndexActionExecutor.java ## @@ -0,0 +1,189 @@ +/* + * Licens

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835767529 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/index/RunIndexActionExecutor.java ## @@ -0,0 +1,263 @@ +/* + * Licens

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835770392 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/index/ScheduleIndexActionExecutor.java ## @@ -0,0 +1,117 @@ +/* + * L

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835770502 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/index/ScheduleIndexActionExecutor.java ## @@ -0,0 +1,138 @@ +/* + * L

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835770935 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/HoodieSparkCopyOnWriteTable.java ## @@ -343,6 +347,16 @@ public HoodieRollbac

[jira] [Created] (HUDI-3719) High performance costs of AvroSerializer in Datasource writing

Tao Meng created HUDI-3719: -- Summary: High performance costs of AvroSerializer in Datasource writing Key: HUDI-3719 URL: https://issues.apache.org/jira/browse/HUDI-3719 Project: Apache Hudi Issue T

[GitHub] [hudi] xiarixiaoyao opened a new pull request #5137: [HUDI-3719] High performance costs of AvroSerizlizer in DataSource wr…

xiarixiaoyao opened a new pull request #5137: URL: https://github.com/apache/hudi/pull/5137 …iting ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ##

[jira] [Updated] (HUDI-3719) High performance costs of AvroSerializer in Datasource writing

[ https://issues.apache.org/jira/browse/HUDI-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3719: - Labels: pull-request-available (was: ) > High performance costs of AvroSerializer in Datasource w

[GitHub] [hudi] hudi-bot commented on pull request #5137: [HUDI-3719] High performance costs of AvroSerizlizer in DataSource wr…

hudi-bot commented on pull request #5137: URL: https://github.com/apache/hudi/pull/5137#issuecomment-1079709263 ## CI report: * 6d18b04edbbaa9a8f8e3645e00df7fa6d76f51e7 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot removed a comment on pull request #5137: [HUDI-3719] High performance costs of AvroSerizlizer in DataSource wr…

hudi-bot removed a comment on pull request #5137: URL: https://github.com/apache/hudi/pull/5137#issuecomment-1079709263 ## CI report: * 6d18b04edbbaa9a8f8e3645e00df7fa6d76f51e7 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #5137: [HUDI-3719] High performance costs of AvroSerizlizer in DataSource wr…

hudi-bot commented on pull request #5137: URL: https://github.com/apache/hudi/pull/5137#issuecomment-1079709771 ## CI report: * 6d18b04edbbaa9a8f8e3645e00df7fa6d76f51e7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835773830 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/metadata/SparkHoodieBackedTableMetadataWriter.java ## @@ -121,6 +121,15 @@ protecte

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835773987 ## File path: hudi-common/src/main/avro/HoodieIndexPartitionInfo.avsc ## @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835774040 ## File path: hudi-common/src/main/avro/HoodieIndexPartitionInfo.avsc ## @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [hudi] xiarixiaoyao commented on pull request #5137: [HUDI-3719] High performance costs of AvroSerizlizer in DataSource wr…

xiarixiaoyao commented on pull request #5137: URL: https://github.com/apache/hudi/pull/5137#issuecomment-1079711532 @xushiyan @YannByron @leesf @alexeykudinkin could you pls help me review this pr, thanks its a serious bug before pacth: 295553 ms after patch: 5279 ms `

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835775436 ## File path: hudi-common/src/main/java/org/apache/hudi/common/bloom/BloomFilter.java ## @@ -30,6 +34,13 @@ */ void add(String key); + /** + * Ad

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835775436 ## File path: hudi-common/src/main/java/org/apache/hudi/common/bloom/BloomFilter.java ## @@ -30,6 +34,13 @@ */ void add(String key); + /** + * Ad

[GitHub] [hudi] xushiyan commented on issue #5107: [SUPPORT] High performance costs of AvroSerializer in Datasource writing

xushiyan commented on issue #5107: URL: https://github.com/apache/hudi/issues/5107#issuecomment-1079713821 @YuweiXiao @qjqqyy @boneanxs @danny0405 thank you folks for raising and the discussion. @xiarixiaoyao has kindly helped verified the fix. -- This is an automated message from the Ap

[GitHub] [hudi] codope commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

codope commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835776000 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java ## @@ -208,6 +208,18 @@ .sinceVersion("0.11.0") .wi

[GitHub] [hudi] nsivabalan commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

nsivabalan commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835776515 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java ## @@ -608,7 +624,7 @@ private v

[GitHub] [hudi] nsivabalan commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

nsivabalan commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835776580 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java ## @@ -608,7 +624,7 @@ private v

[GitHub] [hudi] nsivabalan commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

nsivabalan commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835776905 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java ## @@ -659,20 +691,100 @@ privat

[GitHub] [hudi] nsivabalan commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

nsivabalan commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835776948 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java ## @@ -659,20 +691,100 @@ privat

[GitHub] [hudi] hudi-bot commented on pull request #4945: [HUDI-3538] Support Compaction Command Based on Call Procedure Command for Spark SQL

hudi-bot commented on pull request #4945: URL: https://github.com/apache/hudi/pull/4945#issuecomment-1079717858 ## CI report: * ce55ea05eb9d27917fce72a3e40678ee8a5ed55e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4945: [HUDI-3538] Support Compaction Command Based on Call Procedure Command for Spark SQL

hudi-bot removed a comment on pull request #4945: URL: https://github.com/apache/hudi/pull/4945#issuecomment-1079650411 ## CI report: * ce55ea05eb9d27917fce72a3e40678ee8a5ed55e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4945: [HUDI-3538] Support Compaction Command Based on Call Procedure Command for Spark SQL

hudi-bot commented on pull request #4945: URL: https://github.com/apache/hudi/pull/4945#issuecomment-1079718404 ## CI report: * ce55ea05eb9d27917fce72a3e40678ee8a5ed55e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4945: [HUDI-3538] Support Compaction Command Based on Call Procedure Command for Spark SQL

hudi-bot removed a comment on pull request #4945: URL: https://github.com/apache/hudi/pull/4945#issuecomment-1079717858 ## CI report: * ce55ea05eb9d27917fce72a3e40678ee8a5ed55e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] nsivabalan commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

nsivabalan commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835778181 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieIndexer.java ## @@ -0,0 +1,276 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [hudi] nsivabalan commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

nsivabalan commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835778402 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieIndexer.java ## @@ -0,0 +1,276 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [hudi] nsivabalan commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

nsivabalan commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835778602 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieIndexer.java ## @@ -0,0 +1,276 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [hudi] nsivabalan commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

nsivabalan commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835779230 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java ## @@ -392,6 +398,12 @@ public v

[GitHub] [hudi] nsivabalan commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

nsivabalan commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835779326 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java ## @@ -392,6 +398,12 @@ public v

[GitHub] [hudi] nsivabalan commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

nsivabalan commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835779404 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java ## @@ -392,6 +398,12 @@ public v

[GitHub] [hudi] nsivabalan commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

nsivabalan commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835779585 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/index/ScheduleIndexActionExecutor.java ## @@ -0,0 +1,138 @@ +/* +

[GitHub] [hudi] nsivabalan commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

nsivabalan commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835780780 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/index/RunIndexActionExecutor.java ## @@ -0,0 +1,263 @@ +/* + * Li

[GitHub] [hudi] nsivabalan commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

nsivabalan commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835780906 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/index/RunIndexActionExecutor.java ## @@ -0,0 +1,263 @@ +/* + * Li

[GitHub] [hudi] nsivabalan commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

nsivabalan commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835781695 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/index/RunIndexActionExecutor.java ## @@ -0,0 +1,263 @@ +/* + * Li

[jira] [Updated] (HUDI-3495) Reading keys in parallel from HoodieMetadataMergedLogRecordReader may lead to empty results even if key exists

[ https://issues.apache.org/jira/browse/HUDI-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3495: - Reviewers: Ethan Guo > Reading keys in parallel from HoodieMetadataMergedLogRecordReader may lead to > em

[GitHub] [hudi] nsivabalan commented on a change in pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

nsivabalan commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r835782128 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieIndexer.java ## @@ -0,0 +1,276 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [hudi] hudi-bot removed a comment on pull request #5137: [HUDI-3719] High performance costs of AvroSerizlizer in DataSource wr…

hudi-bot removed a comment on pull request #5137: URL: https://github.com/apache/hudi/pull/5137#issuecomment-1079709771 ## CI report: * 6d18b04edbbaa9a8f8e3645e00df7fa6d76f51e7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #5137: [HUDI-3719] High performance costs of AvroSerizlizer in DataSource wr…

hudi-bot commented on pull request #5137: URL: https://github.com/apache/hudi/pull/5137#issuecomment-1079726338 ## CI report: * 6d18b04edbbaa9a8f8e3645e00df7fa6d76f51e7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

  1   2   >