[GitHub] [hudi] hudi-bot commented on pull request #4721: [HUDI-3320] Hoodie metadata table validator

hudi-bot commented on pull request #4721: URL: https://github.com/apache/hudi/pull/4721#issuecomment-1028704992 ## CI report: * 86b335a77fb92cf34cb5e653694be626f6c7eba4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4721: [HUDI-3320] Hoodie metadata table validator

hudi-bot removed a comment on pull request #4721: URL: https://github.com/apache/hudi/pull/4721#issuecomment-1028678659 ## CI report: * 86b335a77fb92cf34cb5e653694be626f6c7eba4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

hudi-bot removed a comment on pull request #4352: URL: https://github.com/apache/hudi/pull/4352#issuecomment-1028676669 ## CI report: * 235981abd20a498a3e29e98ce0eda9de35018f99 UNKNOWN * d78e61c34acac7e23477f196388076cdd822dd69 Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot commented on pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

hudi-bot commented on pull request #4352: URL: https://github.com/apache/hudi/pull/4352#issuecomment-1028707127 ## CI report: * 235981abd20a498a3e29e98ce0eda9de35018f99 UNKNOWN * d78e61c34acac7e23477f196388076cdd822dd69 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4721: [HUDI-3320] Hoodie metadata table validator

hudi-bot removed a comment on pull request #4721: URL: https://github.com/apache/hudi/pull/4721#issuecomment-1028704992 ## CI report: * 86b335a77fb92cf34cb5e653694be626f6c7eba4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4721: [HUDI-3320] Hoodie metadata table validator

hudi-bot commented on pull request #4721: URL: https://github.com/apache/hudi/pull/4721#issuecomment-1028707663 ## CI report: * 540459a7e5a71f01b8e052424ccabad5a25b840e Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #4721: [HUDI-3320] Hoodie metadata table validator

hudi-bot removed a comment on pull request #4721: URL: https://github.com/apache/hudi/pull/4721#issuecomment-1028707663 ## CI report: * 540459a7e5a71f01b8e052424ccabad5a25b840e Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4721: [HUDI-3320] Hoodie metadata table validator

hudi-bot commented on pull request #4721: URL: https://github.com/apache/hudi/pull/4721#issuecomment-1028719892 ## CI report: * 540459a7e5a71f01b8e052424ccabad5a25b840e Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] zhangyue19921010 commented on a change in pull request #4721: [HUDI-3320] Hoodie metadata table validator

zhangyue19921010 commented on a change in pull request #4721: URL: https://github.com/apache/hudi/pull/4721#discussion_r798336089 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java ## @@ -0,0 +1,458 @@ +/* + * Licensed to the

[GitHub] [hudi] manojpec opened a new pull request #4740: [HUDI-3356][HUDI-3203] HoodieData for metadata index records; BloomFilter construction from index based on the type param

manojpec opened a new pull request #4740: URL: https://github.com/apache/hudi/pull/4740 ## What is the purpose of the pull request - Write stats are converted to metadata index records during the commit. Making them use the HoodieData type so that the record generation sc

[jira] [Updated] (HUDI-3356) Conversion of write stats to metadata index records should use HoodieData throughout

[ https://issues.apache.org/jira/browse/HUDI-3356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3356: - Labels: pull-request-available (was: ) > Conversion of write stats to metadata index records shou

[GitHub] [hudi] hudi-bot commented on pull request #4740: [HUDI-1295][HUDI-3356][HUDI-3203] HoodieData for metadata index records; BloomFilter construction from index based on the type param

hudi-bot commented on pull request #4740: URL: https://github.com/apache/hudi/pull/4740#issuecomment-1028763526 ## CI report: * b7079f6cad2f05c32bb14f5589735d25aaae4072 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot commented on pull request #4740: [HUDI-1295][HUDI-3356][HUDI-3203] HoodieData for metadata index records; BloomFilter construction from index based on the type param

hudi-bot commented on pull request #4740: URL: https://github.com/apache/hudi/pull/4740#issuecomment-1028765967 ## CI report: * b7079f6cad2f05c32bb14f5589735d25aaae4072 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4740: [HUDI-1295][HUDI-3356][HUDI-3203] HoodieData for metadata index records; BloomFilter construction from index based on the type param

hudi-bot removed a comment on pull request #4740: URL: https://github.com/apache/hudi/pull/4740#issuecomment-1028763526 ## CI report: * b7079f6cad2f05c32bb14f5589735d25aaae4072 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

hudi-bot commented on pull request #4352: URL: https://github.com/apache/hudi/pull/4352#issuecomment-1028767916 ## CI report: * 235981abd20a498a3e29e98ce0eda9de35018f99 UNKNOWN * e48904554517673727ee5a0cb4055579f39e Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

hudi-bot removed a comment on pull request #4352: URL: https://github.com/apache/hudi/pull/4352#issuecomment-1028707127 ## CI report: * 235981abd20a498a3e29e98ce0eda9de35018f99 UNKNOWN * d78e61c34acac7e23477f196388076cdd822dd69 Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot removed a comment on pull request #4721: [HUDI-3320] Hoodie metadata table validator

hudi-bot removed a comment on pull request #4721: URL: https://github.com/apache/hudi/pull/4721#issuecomment-1028719892 ## CI report: * 540459a7e5a71f01b8e052424ccabad5a25b840e Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4721: [HUDI-3320] Hoodie metadata table validator

hudi-bot commented on pull request #4721: URL: https://github.com/apache/hudi/pull/4721#issuecomment-1028789883 ## CI report: * 06a8178eb64bdd7f40dd426b9afb1746797dea46 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #4740: [HUDI-1295][HUDI-3356][HUDI-3203] HoodieData for metadata index records; BloomFilter construction from index based on the type param

hudi-bot commented on pull request #4740: URL: https://github.com/apache/hudi/pull/4740#issuecomment-1028826855 ## CI report: * b7079f6cad2f05c32bb14f5589735d25aaae4072 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4740: [HUDI-1295][HUDI-3356][HUDI-3203] HoodieData for metadata index records; BloomFilter construction from index based on the type param

hudi-bot removed a comment on pull request #4740: URL: https://github.com/apache/hudi/pull/4740#issuecomment-1028765967 ## CI report: * b7079f6cad2f05c32bb14f5589735d25aaae4072 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] YannByron commented on issue #4552: [BUG] Data corrupted in the timestamp field to 1970-01-01 19:45:30.000 after subsequent upsert run

YannByron commented on issue #4552: URL: https://github.com/apache/hudi/issues/4552#issuecomment-1028827672 @denys-tyshetskyy @rafcis02 This issue, i think, is a bug for Bulk_Insert and `hoodie.datasource.write.row.writer.enable`. But for now, you can disable `hoodie.datasource.write

[GitHub] [hudi] t0il3ts0ap opened a new pull request #4741: [MINOR] Add configuration to enable/disable common prefix for all met…

t0il3ts0ap opened a new pull request #4741: URL: https://github.com/apache/hudi/pull/4741 …rics ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## Wha

[GitHub] [hudi] hudi-bot commented on pull request #4741: [MINOR] Add configuration to enable/disable common prefix for all met…

hudi-bot commented on pull request #4741: URL: https://github.com/apache/hudi/pull/4741#issuecomment-1028834445 ## CI report: * 347641d0cef968cb368ec7ebd073316026a82af7 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot removed a comment on pull request #4741: [MINOR] Add configuration to enable/disable common prefix for all met…

hudi-bot removed a comment on pull request #4741: URL: https://github.com/apache/hudi/pull/4741#issuecomment-1028834445 ## CI report: * 347641d0cef968cb368ec7ebd073316026a82af7 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4741: [MINOR] Add configuration to enable/disable common prefix for all met…

hudi-bot commented on pull request #4741: URL: https://github.com/apache/hudi/pull/4741#issuecomment-1028837130 ## CI report: * 347641d0cef968cb368ec7ebd073316026a82af7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] YannByron commented on issue #4678: [SUPPORT] spark.read.format("hudi").schema(userSpecifiedSchema) doesn't work in version 0.10.0 ,but does work in 0.5.3

YannByron commented on issue #4678: URL: https://github.com/apache/hudi/issues/4678#issuecomment-1028852790 @wjcwin can you give some details like codes and your expected result. that will help us to locate this issue. -- This is an automated message from the Apache Git Service. To respo

[GitHub] [hudi] hudi-bot commented on pull request #4741: [MINOR] Add configuration to enable/disable common prefix for all met…

hudi-bot commented on pull request #4741: URL: https://github.com/apache/hudi/pull/4741#issuecomment-1028884413 ## CI report: * 347641d0cef968cb368ec7ebd073316026a82af7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4741: [MINOR] Add configuration to enable/disable common prefix for all met…

hudi-bot removed a comment on pull request #4741: URL: https://github.com/apache/hudi/pull/4741#issuecomment-1028837130 ## CI report: * 347641d0cef968cb368ec7ebd073316026a82af7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] YannByron commented on a change in pull request #4714: [HUDI-3204] fix problem that spark on TimestampKeyGenerator has no re…

YannByron commented on a change in pull request #4714: URL: https://github.com/apache/hudi/pull/4714#discussion_r798474157 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java ## @@ -183,6 +185,14 @@ public static final ConfigProperty

[GitHub] [hudi] YannByron commented on a change in pull request #4714: [HUDI-3204] fix problem that spark on TimestampKeyGenerator has no re…

YannByron commented on a change in pull request #4714: URL: https://github.com/apache/hudi/pull/4714#discussion_r798478214 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableMetaClient.java ## @@ -759,6 +780,11 @@ public PropertyBuilder fromMetaCl

[GitHub] [hudi] YannByron commented on a change in pull request #4714: [HUDI-3204] fix problem that spark on TimestampKeyGenerator has no re…

YannByron commented on a change in pull request #4714: URL: https://github.com/apache/hudi/pull/4714#discussion_r798482835 ## File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieFileIndex.scala ## @@ -282,6 +296,41 @@ object HoodieFileIndex

[GitHub] [hudi] nsivabalan commented on pull request #4420: [HUDI-1847] Adding inline scheduling support for spark datasource path for compaction and clustering

nsivabalan commented on pull request #4420: URL: https://github.com/apache/hudi/pull/4420#issuecomment-1028912096 wrt variables names, I kept it in sync w/ existing ones. for eg, config for inline compaction is called "hoodie.compact.inline", but variable is named as INLINE_COMPACT. Gu

[GitHub] [hudi] YannByron commented on a change in pull request #4714: [HUDI-3204] fix problem that spark on TimestampKeyGenerator has no re…

YannByron commented on a change in pull request #4714: URL: https://github.com/apache/hudi/pull/4714#discussion_r798492743 ## File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieFileIndex.scala ## @@ -282,6 +296,41 @@ object HoodieFileIndex

[GitHub] [hudi] nsivabalan commented on a change in pull request #4420: [HUDI-1847] Adding inline scheduling support for spark datasource path for compaction and clustering

nsivabalan commented on a change in pull request #4420: URL: https://github.com/apache/hudi/pull/4420#discussion_r784786957 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieClusteringConfig.java ## @@ -143,6 +143,12 @@ .withDocumen

[GitHub] [hudi] nsivabalan commented on a change in pull request #4681: [HUDI-2987] Update all deprecated calls to new apis in HoodieRecordPayload

nsivabalan commented on a change in pull request #4681: URL: https://github.com/apache/hudi/pull/4681#discussion_r798499028 ## File path: hudi-common/src/main/java/org/apache/hudi/common/model/debezium/PostgresDebeziumAvroPayload.java ## @@ -71,6 +72,19 @@ protected boolean sh

[GitHub] [hudi] hudi-bot removed a comment on pull request #4420: [HUDI-1847] Adding inline scheduling support for spark datasource path for compaction and clustering

hudi-bot removed a comment on pull request #4420: URL: https://github.com/apache/hudi/pull/4420#issuecomment-1026975832 ## CI report: * d9e9f98b3025b2245d2e0ca6f54eca86b5286e3e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4420: [HUDI-1847] Adding inline scheduling support for spark datasource path for compaction and clustering

hudi-bot commented on pull request #4420: URL: https://github.com/apache/hudi/pull/4420#issuecomment-1028929692 ## CI report: * d9e9f98b3025b2245d2e0ca6f54eca86b5286e3e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #4681: [HUDI-2987] Update all deprecated calls to new apis in HoodieRecordPayload

hudi-bot commented on pull request #4681: URL: https://github.com/apache/hudi/pull/4681#issuecomment-1028930081 ## CI report: * f96dd027453767ff6c76efdf55c12ced7befa008 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4681: [HUDI-2987] Update all deprecated calls to new apis in HoodieRecordPayload

hudi-bot removed a comment on pull request #4681: URL: https://github.com/apache/hudi/pull/4681#issuecomment-1024690651 ## CI report: * f96dd027453767ff6c76efdf55c12ced7befa008 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] YannByron commented on a change in pull request #4714: [HUDI-3204] fix problem that spark on TimestampKeyGenerator has no re…

YannByron commented on a change in pull request #4714: URL: https://github.com/apache/hudi/pull/4714#discussion_r798492743 ## File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieFileIndex.scala ## @@ -282,6 +296,41 @@ object HoodieFileIndex

[GitHub] [hudi] hudi-bot removed a comment on pull request #4420: [HUDI-1847] Adding inline scheduling support for spark datasource path for compaction and clustering

hudi-bot removed a comment on pull request #4420: URL: https://github.com/apache/hudi/pull/4420#issuecomment-1028929692 ## CI report: * d9e9f98b3025b2245d2e0ca6f54eca86b5286e3e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4420: [HUDI-1847] Adding inline scheduling support for spark datasource path for compaction and clustering

hudi-bot commented on pull request #4420: URL: https://github.com/apache/hudi/pull/4420#issuecomment-1028932141 ## CI report: * d9e9f98b3025b2245d2e0ca6f54eca86b5286e3e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #4681: [HUDI-2987] Update all deprecated calls to new apis in HoodieRecordPayload

hudi-bot commented on pull request #4681: URL: https://github.com/apache/hudi/pull/4681#issuecomment-1028932513 ## CI report: * f96dd027453767ff6c76efdf55c12ced7befa008 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4681: [HUDI-2987] Update all deprecated calls to new apis in HoodieRecordPayload

hudi-bot removed a comment on pull request #4681: URL: https://github.com/apache/hudi/pull/4681#issuecomment-1028930081 ## CI report: * f96dd027453767ff6c76efdf55c12ced7befa008 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4681: [HUDI-2987] Update all deprecated calls to new apis in HoodieRecordPayload

hudi-bot commented on pull request #4681: URL: https://github.com/apache/hudi/pull/4681#issuecomment-1028935076 ## CI report: * 8c7a3e95657431424cde73e7c7185db4fea92085 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #4714: [HUDI-3204] fix problem that spark on TimestampKeyGenerator has no re…

hudi-bot commented on pull request #4714: URL: https://github.com/apache/hudi/pull/4714#issuecomment-1028935199 ## CI report: * e8999e4928debb876332f287a1584cc7cbd69c85 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4681: [HUDI-2987] Update all deprecated calls to new apis in HoodieRecordPayload

hudi-bot removed a comment on pull request #4681: URL: https://github.com/apache/hudi/pull/4681#issuecomment-1028932513 ## CI report: * f96dd027453767ff6c76efdf55c12ced7befa008 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4714: [HUDI-3204] fix problem that spark on TimestampKeyGenerator has no re…

hudi-bot removed a comment on pull request #4714: URL: https://github.com/apache/hudi/pull/4714#issuecomment-1024850151 ## CI report: * e8999e4928debb876332f287a1584cc7cbd69c85 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4714: [HUDI-3204] fix problem that spark on TimestampKeyGenerator has no re…

hudi-bot commented on pull request #4714: URL: https://github.com/apache/hudi/pull/4714#issuecomment-1028937703 ## CI report: * e8999e4928debb876332f287a1584cc7cbd69c85 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4714: [HUDI-3204] fix problem that spark on TimestampKeyGenerator has no re…

hudi-bot removed a comment on pull request #4714: URL: https://github.com/apache/hudi/pull/4714#issuecomment-1028935199 ## CI report: * e8999e4928debb876332f287a1584cc7cbd69c85 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[jira] [Updated] (HUDI-3367) Add spark scheduling configs support to spark data source writer

[ https://issues.apache.org/jira/browse/HUDI-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3367: -- Fix Version/s: 0.11.0 > Add spark scheduling configs support to spark data source writer

[jira] [Created] (HUDI-3367) Add spark scheduling configs support to spark data source writer

sivabalan narayanan created HUDI-3367: - Summary: Add spark scheduling configs support to spark data source writer Key: HUDI-3367 URL: https://issues.apache.org/jira/browse/HUDI-3367 Project: Apach

[jira] [Updated] (HUDI-3367) Add spark scheduling configs support to spark data source writer

[ https://issues.apache.org/jira/browse/HUDI-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3367: -- Priority: Blocker (was: Major) > Add spark scheduling configs support to spark data sou

[jira] [Assigned] (HUDI-3367) Add spark scheduling configs support to spark data source writer

[ https://issues.apache.org/jira/browse/HUDI-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-3367: - Assignee: sivabalan narayanan > Add spark scheduling configs support to spark dat

[GitHub] [hudi] codope merged pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

codope merged pull request #4352: URL: https://github.com/apache/hudi/pull/4352 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr..

[hudi] branch master updated (d681824 -> 5927bdd)

This is an automated email from the ASF dual-hosted git repository. codope pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from d681824 [HUDI-3337] Fixing Parquet Column Range metadata extraction (#4705) add 5927bdd [HUDI-1295] Metadata In

[GitHub] [hudi] hudi-bot commented on pull request #4659: [HUDI-3091] Making SIMPLE index as the default index type

hudi-bot commented on pull request #4659: URL: https://github.com/apache/hudi/pull/4659#issuecomment-1028958399 ## CI report: * 647c6a4d9dad7e517c48d70857f9ebd2faf5c57c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4659: [HUDI-3091] Making SIMPLE index as the default index type

hudi-bot removed a comment on pull request #4659: URL: https://github.com/apache/hudi/pull/4659#issuecomment-1028504428 ## CI report: * 647c6a4d9dad7e517c48d70857f9ebd2faf5c57c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4659: [HUDI-3091] Making SIMPLE index as the default index type

hudi-bot commented on pull request #4659: URL: https://github.com/apache/hudi/pull/4659#issuecomment-1028961438 ## CI report: * 647c6a4d9dad7e517c48d70857f9ebd2faf5c57c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4659: [HUDI-3091] Making SIMPLE index as the default index type

hudi-bot removed a comment on pull request #4659: URL: https://github.com/apache/hudi/pull/4659#issuecomment-1028958399 ## CI report: * 647c6a4d9dad7e517c48d70857f9ebd2faf5c57c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] codope commented on a change in pull request #4741: [MINOR] Add configuration to enable/disable common prefix for all met…

codope commented on a change in pull request #4741: URL: https://github.com/apache/hudi/pull/4741#discussion_r798534340 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/metrics/HoodieMetricsConfig.java ## @@ -84,6 +84,13 @@ .sinceVersion

[GitHub] [hudi] hudi-bot commented on pull request #4714: [HUDI-3204] fix problem that spark on TimestampKeyGenerator has no re…

hudi-bot commented on pull request #4714: URL: https://github.com/apache/hudi/pull/4714#issuecomment-1028974180 ## CI report: * 8ee6a4e07374f18a35f3f1807b11fa5b6570cb6e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4714: [HUDI-3204] fix problem that spark on TimestampKeyGenerator has no re…

hudi-bot removed a comment on pull request #4714: URL: https://github.com/apache/hudi/pull/4714#issuecomment-1028937703 ## CI report: * e8999e4928debb876332f287a1584cc7cbd69c85 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[jira] [Updated] (HUDI-3362) Hudi 0.8.0 cannot roleback CoW table

[ https://issues.apache.org/jira/browse/HUDI-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Darvas updated HUDI-3362: Summary: Hudi 0.8.0 cannot roleback CoW table (was: Hudi 0.8.0 cannot roleback MoR table) > Hudi 0

[jira] [Commented] (HUDI-3362) Hudi 0.8.0 cannot roleback CoW table

[ https://issues.apache.org/jira/browse/HUDI-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486463#comment-17486463 ] Istvan Darvas commented on HUDI-3362: - I have changed MoR => CoW it was a mistake...Yo

[GitHub] [hudi] hudi-bot commented on pull request #4420: [HUDI-1847] Adding inline scheduling support for spark datasource path for compaction and clustering

hudi-bot commented on pull request #4420: URL: https://github.com/apache/hudi/pull/4420#issuecomment-1028987325 ## CI report: * 48fd6ddd0a36975f7e065ae3c4c4635422def062 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4420: [HUDI-1847] Adding inline scheduling support for spark datasource path for compaction and clustering

hudi-bot removed a comment on pull request #4420: URL: https://github.com/apache/hudi/pull/4420#issuecomment-1028932141 ## CI report: * d9e9f98b3025b2245d2e0ca6f54eca86b5286e3e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[jira] [Updated] (HUDI-3367) Add spark scheduling configs support to spark streaming writes

[ https://issues.apache.org/jira/browse/HUDI-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3367: -- Summary: Add spark scheduling configs support to spark streaming writes (was: Add spark

[jira] [Updated] (HUDI-3367) Add spark scheduling configs support to spark streaming writes

[ https://issues.apache.org/jira/browse/HUDI-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3367: -- Description: We have support to set scheduling configs in spark for deltastreamer. this

[jira] [Updated] (HUDI-3362) Hudi 0.8.0 cannot roleback CoW table

[ https://issues.apache.org/jira/browse/HUDI-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Darvas updated HUDI-3362: Attachment: rollback-on-a-not-damaged-table-SUCCESS.txt > Hudi 0.8.0 cannot roleback CoW table > ---

[jira] [Commented] (HUDI-3362) Hudi 0.8.0 cannot roleback CoW table

[ https://issues.apache.org/jira/browse/HUDI-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486486#comment-17486486 ] Istvan Darvas commented on HUDI-3362: - I was asked to do this:   and if you have a ta

[jira] [Updated] (HUDI-3362) Hudi 0.8.0 cannot roleback CoW table

[ https://issues.apache.org/jira/browse/HUDI-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Darvas updated HUDI-3362: Attachment: rollback-on-a-not-damaged-table-SUCCESS.pdf > Hudi 0.8.0 cannot roleback CoW table > ---

[GitHub] [hudi] hudi-bot commented on pull request #4659: [HUDI-3091] Making SIMPLE index as the default index type

hudi-bot commented on pull request #4659: URL: https://github.com/apache/hudi/pull/4659#issuecomment-1029063101 ## CI report: * c4d455ef4ee1ba207336c7af33400d33c7bf1f02 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4659: [HUDI-3091] Making SIMPLE index as the default index type

hudi-bot removed a comment on pull request #4659: URL: https://github.com/apache/hudi/pull/4659#issuecomment-1028961438 ## CI report: * 647c6a4d9dad7e517c48d70857f9ebd2faf5c57c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] nsivabalan opened a new pull request #4742: [HUDI-3367] Adding scheduling configs support to spark streaming

nsivabalan opened a new pull request #4742: URL: https://github.com/apache/hudi/pull/4742 ## What is the purpose of the pull request - Adding support to set scheduling configs to spark streaming writes. With async compaction and spark streaming writes, users can set diff weights and

[jira] [Updated] (HUDI-3362) Hudi 0.8.0 cannot roleback CoW table

[ https://issues.apache.org/jira/browse/HUDI-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Darvas updated HUDI-3362: Description: Hi Guys,   Environment: AWS EMR 6.4 / Hudi v0.8.0 Problem: I have a CoW table wich i

[jira] [Updated] (HUDI-3367) Add spark scheduling configs support to spark streaming writes

[ https://issues.apache.org/jira/browse/HUDI-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3367: - Labels: pull-request-available (was: ) > Add spark scheduling configs support to spark streaming

[jira] [Updated] (HUDI-3362) Hudi 0.8.0 cannot roleback CoW table

[ https://issues.apache.org/jira/browse/HUDI-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Darvas updated HUDI-3362: Description: Hi Guys,   Environment: AWS EMR 6.4 / Hudi v0.8.0 Problem: I have a CoW table wich i

[GitHub] [hudi] hudi-bot commented on pull request #4742: [HUDI-3367] Adding scheduling configs support to spark streaming

hudi-bot commented on pull request #4742: URL: https://github.com/apache/hudi/pull/4742#issuecomment-1029141912 ## CI report: * d9cc2e405254369a5bdd65fe0c27d9b6c8eaf1c7 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot commented on pull request #4742: [HUDI-3367] Adding scheduling configs support to spark streaming

hudi-bot commented on pull request #4742: URL: https://github.com/apache/hudi/pull/4742#issuecomment-1029145207 ## CI report: * d9cc2e405254369a5bdd65fe0c27d9b6c8eaf1c7 UNKNOWN * 2dd12aacf0c0f8a203f2f2bf195fda423160a7a8 UNKNOWN Bot commands @hudi-bot supports th

[GitHub] [hudi] hudi-bot removed a comment on pull request #4742: [HUDI-3367] Adding scheduling configs support to spark streaming

hudi-bot removed a comment on pull request #4742: URL: https://github.com/apache/hudi/pull/4742#issuecomment-1029141912 ## CI report: * d9cc2e405254369a5bdd65fe0c27d9b6c8eaf1c7 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4742: [HUDI-3367] Adding scheduling configs support to spark streaming

hudi-bot commented on pull request #4742: URL: https://github.com/apache/hudi/pull/4742#issuecomment-1029148478 ## CI report: * d9cc2e405254369a5bdd65fe0c27d9b6c8eaf1c7 UNKNOWN * 2dd12aacf0c0f8a203f2f2bf195fda423160a7a8 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4742: [HUDI-3367] Adding scheduling configs support to spark streaming

hudi-bot removed a comment on pull request #4742: URL: https://github.com/apache/hudi/pull/4742#issuecomment-1029145207 ## CI report: * d9cc2e405254369a5bdd65fe0c27d9b6c8eaf1c7 UNKNOWN * 2dd12aacf0c0f8a203f2f2bf195fda423160a7a8 UNKNOWN Bot commands @hudi-bot sup

[jira] [Updated] (HUDI-3362) Hudi 0.8.0 cannot roleback CoW table

[ https://issues.apache.org/jira/browse/HUDI-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Darvas updated HUDI-3362: Attachment: hoodie-list.txt > Hudi 0.8.0 cannot roleback CoW table > ---

[jira] [Comment Edited] (HUDI-3362) Hudi 0.8.0 cannot roleback CoW table

[ https://issues.apache.org/jira/browse/HUDI-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486594#comment-17486594 ] Istvan Darvas edited comment on HUDI-3362 at 2/3/22, 4:53 PM: --

[jira] [Commented] (HUDI-3362) Hudi 0.8.0 cannot roleback CoW table

[ https://issues.apache.org/jira/browse/HUDI-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486594#comment-17486594 ] Istvan Darvas commented on HUDI-3362: - So with 0.8.0 i was able to restore over a succ

[jira] [Comment Edited] (HUDI-3362) Hudi 0.8.0 cannot roleback CoW table

[ https://issues.apache.org/jira/browse/HUDI-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486594#comment-17486594 ] Istvan Darvas edited comment on HUDI-3362 at 2/3/22, 4:54 PM: --

[jira] [Updated] (HUDI-3362) Hudi 0.8.0 cannot roleback CoW table

[ https://issues.apache.org/jira/browse/HUDI-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Darvas updated HUDI-3362: Attachment: rollback-on-a-not-damaged-table-over-replace-commit.pdf > Hudi 0.8.0 cannot roleback CoW

[GitHub] [hudi] hudi-bot commented on pull request #4742: [HUDI-3367] Adding scheduling configs support to spark streaming

hudi-bot commented on pull request #4742: URL: https://github.com/apache/hudi/pull/4742#issuecomment-1029212546 ## CI report: * d9cc2e405254369a5bdd65fe0c27d9b6c8eaf1c7 UNKNOWN * 2dd12aacf0c0f8a203f2f2bf195fda423160a7a8 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4742: [HUDI-3367] Adding scheduling configs support to spark streaming

hudi-bot removed a comment on pull request #4742: URL: https://github.com/apache/hudi/pull/4742#issuecomment-1029148478 ## CI report: * d9cc2e405254369a5bdd65fe0c27d9b6c8eaf1c7 UNKNOWN * 2dd12aacf0c0f8a203f2f2bf195fda423160a7a8 Azure: [PENDING](https://dev.azure.com/apache-hud

[jira] [Commented] (HUDI-2928) Evaluate rebasing Hudi's default compression from Gzip to Zstd

[ https://issues.apache.org/jira/browse/HUDI-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486622#comment-17486622 ] Alexey Kudinkin commented on HUDI-2928: --- Uber's example of leveraging Zstd in lieu o

[GitHub] [hudi] hudi-bot removed a comment on pull request #4693: [WIP][HUDI-3175][RFC-45] Implement async metadata indexing

hudi-bot removed a comment on pull request #4693: URL: https://github.com/apache/hudi/pull/4693#issuecomment-1025615583 ## CI report: * c5c563ffa6625d610c9c6bd252457129ce5ccddc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4693: [WIP][HUDI-3175][RFC-45] Implement async metadata indexing

hudi-bot commented on pull request #4693: URL: https://github.com/apache/hudi/pull/4693#issuecomment-1029256806 ## CI report: * c5c563ffa6625d610c9c6bd252457129ce5ccddc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4693: [WIP][HUDI-3175][RFC-45] Implement async metadata indexing

hudi-bot removed a comment on pull request #4693: URL: https://github.com/apache/hudi/pull/4693#issuecomment-1029256806 ## CI report: * c5c563ffa6625d610c9c6bd252457129ce5ccddc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4693: [WIP][HUDI-3175][RFC-45] Implement async metadata indexing

hudi-bot commented on pull request #4693: URL: https://github.com/apache/hudi/pull/4693#issuecomment-1029259900 ## CI report: * c5c563ffa6625d610c9c6bd252457129ce5ccddc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4711: [HUDI-3166] Hoodie Index Type Metadata Bloom implementation

hudi-bot removed a comment on pull request #4711: URL: https://github.com/apache/hudi/pull/4711#issuecomment-1027577458 ## CI report: * 8f1fa5c06ee5febc006599bac166f9bf3ad98ef7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4711: [HUDI-3166] Hoodie Index Type Metadata Bloom implementation

hudi-bot commented on pull request #4711: URL: https://github.com/apache/hudi/pull/4711#issuecomment-1029270351 ## CI report: * 8f1fa5c06ee5febc006599bac166f9bf3ad98ef7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[jira] [Updated] (HUDI-3166) Implement new HoodieIndex based on metadata indices

[ https://issues.apache.org/jira/browse/HUDI-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3166: - Labels: metadata pull-request-available (was: metadata) > Implement new HoodieIndex based on meta

[GitHub] [hudi] hudi-bot removed a comment on pull request #4711: [HUDI-3166] Hoodie Index Type Metadata Bloom implementation

hudi-bot removed a comment on pull request #4711: URL: https://github.com/apache/hudi/pull/4711#issuecomment-1029270351 ## CI report: * 8f1fa5c06ee5febc006599bac166f9bf3ad98ef7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4711: [HUDI-3166] Hoodie Index Type Metadata Bloom implementation

hudi-bot commented on pull request #4711: URL: https://github.com/apache/hudi/pull/4711#issuecomment-1029272848 ## CI report: * 8f1fa5c06ee5febc006599bac166f9bf3ad98ef7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

  1   2   3   >