[GitHub] [hudi] hudi-bot removed a comment on pull request #4918: [HUDI-3518] Make HiveSchemaProvider support AWS Glue Catalog

hudi-bot removed a comment on pull request #4918: URL: https://github.com/apache/hudi/pull/4918#issuecomment-1068806989 ## CI report: * 4be9c06e21020e9419112cc5402f3b2455c90c73 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4918: [HUDI-3518] Make HiveSchemaProvider support AWS Glue Catalog

hudi-bot commented on pull request #4918: URL: https://github.com/apache/hudi/pull/4918#issuecomment-1070376118 ## CI report: * 4be9c06e21020e9419112cc5402f3b2455c90c73 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] wangxianghu commented on pull request #4918: [HUDI-3518] Make HiveSchemaProvider support AWS Glue Catalog

wangxianghu commented on pull request #4918: URL: https://github.com/apache/hudi/pull/4918#issuecomment-1070375564 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [hudi] hudi-bot commented on pull request #4856: [HUDI-2439] Replace RDD with HoodieData in HoodieSparkTable and commit executors

hudi-bot commented on pull request #4856: URL: https://github.com/apache/hudi/pull/4856#issuecomment-1070374556 ## CI report: * 146e7109d3607a71a97c0f1a9714ca29e7682b97 UNKNOWN * 03718408ced7ed827ea010220b333c807e2eee4a UNKNOWN * 55d361787545f5f51d431e02a64374eb7d475004 UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4856: [HUDI-2439] Replace RDD with HoodieData in HoodieSparkTable and commit executors

hudi-bot removed a comment on pull request #4856: URL: https://github.com/apache/hudi/pull/4856#issuecomment-1066300015 ## CI report: * 146e7109d3607a71a97c0f1a9714ca29e7682b97 UNKNOWN * 03718408ced7ed827ea010220b333c807e2eee4a UNKNOWN * 55d361787545f5f51d431e02a64374eb7d475

[GitHub] [hudi] hudi-bot commented on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

hudi-bot commented on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-1070350319 ## CI report: * 9ba5c351c32f9f364c30b4bc9a814075150d9728 UNKNOWN * 53b4cec7d05dd0bb5a622ffb33597471da7711de UNKNOWN * d8d62a5172a872f65aaca78ee5b3e8bc9c5d3648 Azur

[GitHub] [hudi] hudi-bot removed a comment on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

hudi-bot removed a comment on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-1070319717 ## CI report: * 9ba5c351c32f9f364c30b4bc9a814075150d9728 UNKNOWN * 53b4cec7d05dd0bb5a622ffb33597471da7711de UNKNOWN * 438eecf51f0a2c08270ff6c1bca0ea2e4e83c

[hudi] branch master updated (95e6e53 -> 5ba2d9a)

This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 95e6e53 [HUDI-3404] Automatically adjust write configs based on metadata table and write concurrency mode (#4975)

[GitHub] [hudi] nsivabalan merged pull request #4974: [HUDI-3494] Consider triggering condition of MOR compaction during archival

nsivabalan merged pull request #4974: URL: https://github.com/apache/hudi/pull/4974 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[jira] [Updated] (HUDI-3404) Auto adjust configs for metadata table enablement or turn off metadata table if necessary

[ https://issues.apache.org/jira/browse/HUDI-3404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3404: - Issue Type: Improvement (was: Task) > Auto adjust configs for metadata table enablement or turn off metad

[jira] [Closed] (HUDI-3404) Auto adjust configs for metadata table enablement or turn off metadata table if necessary

[ https://issues.apache.org/jira/browse/HUDI-3404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-3404. Resolution: Done > Auto adjust configs for metadata table enablement or turn off metadata table > if necess

[hudi] branch master updated (8ca9a54 -> 95e6e53)

This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 8ca9a54 [Hudi-3376] Add an option to skip under deletion files for HoodieMetadataTableValidator (#4994) add 9

[GitHub] [hudi] nsivabalan merged pull request #4975: [HUDI-3404] Automatically adjust write configs based on metadata table and write concurrency mode

nsivabalan merged pull request #4975: URL: https://github.com/apache/hudi/pull/4975 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[GitHub] [hudi] nsivabalan commented on pull request #4975: [HUDI-3404] Automatically adjust write configs based on metadata table and write concurrency mode

nsivabalan commented on pull request #4975: URL: https://github.com/apache/hudi/pull/4975#issuecomment-1070339481 Good job on the patch Ethan! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [hudi] hudi-bot removed a comment on pull request #4955: [HUDI-3549][Stacked on 4996][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot removed a comment on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1070243213 ## CI report: * 69b821217672dfff8fe3a3b058a480579e8f5e68 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4955: [HUDI-3549][Stacked on 4996][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot commented on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1070320957 ## CI report: * 5157bbc772c7019ed3f8baaef9618e7d516fde75 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

hudi-bot removed a comment on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-1070311808 ## CI report: * 9ba5c351c32f9f364c30b4bc9a814075150d9728 UNKNOWN * 53b4cec7d05dd0bb5a622ffb33597471da7711de UNKNOWN * 438eecf51f0a2c08270ff6c1bca0ea2e4e83c

[GitHub] [hudi] hudi-bot commented on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

hudi-bot commented on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-1070319717 ## CI report: * 9ba5c351c32f9f364c30b4bc9a814075150d9728 UNKNOWN * 53b4cec7d05dd0bb5a622ffb33597471da7711de UNKNOWN * 438eecf51f0a2c08270ff6c1bca0ea2e4e83c594 Azur

[GitHub] [hudi] hudi-bot commented on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

hudi-bot commented on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-1070311808 ## CI report: * 9ba5c351c32f9f364c30b4bc9a814075150d9728 UNKNOWN * 53b4cec7d05dd0bb5a622ffb33597471da7711de UNKNOWN * 438eecf51f0a2c08270ff6c1bca0ea2e4e83c594 Azur

[GitHub] [hudi] hudi-bot removed a comment on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

hudi-bot removed a comment on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-1069910671 ## CI report: * 9ba5c351c32f9f364c30b4bc9a814075150d9728 UNKNOWN * 53b4cec7d05dd0bb5a622ffb33597471da7711de UNKNOWN * 438eecf51f0a2c08270ff6c1bca0ea2e4e83c

[GitHub] [hudi] hudi-bot removed a comment on pull request #4955: [HUDI-3549][Stacked on 4996][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot removed a comment on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1070236168 ## CI report: * f6e1db77b0b50dcc049c4f051d1bdd62f2942b9a Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4955: [HUDI-3549][Stacked on 4996][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot commented on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1070243213 ## CI report: * 69b821217672dfff8fe3a3b058a480579e8f5e68 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #4955: [HUDI-3549][Stacked on 4996][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot removed a comment on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1070198397 ## CI report: * f6e1db77b0b50dcc049c4f051d1bdd62f2942b9a Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4955: [HUDI-3549][Stacked on 4996][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot commented on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1070236168 ## CI report: * f6e1db77b0b50dcc049c4f051d1bdd62f2942b9a Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot commented on pull request #4955: [HUDI-3549][Stacked on 4996][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot commented on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1070198397 ## CI report: * f6e1db77b0b50dcc049c4f051d1bdd62f2942b9a Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #4955: [HUDI-3549][Stacked on 4996][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot removed a comment on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1070181104 ## CI report: * d4b2b9aa43813e845c10d3284d600caa5a86b819 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4955: [HUDI-3549][Stacked on 4996][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot removed a comment on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1070014635 ## CI report: * d4b2b9aa43813e845c10d3284d600caa5a86b819 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4955: [HUDI-3549][Stacked on 4996][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot commented on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1070181104 ## CI report: * d4b2b9aa43813e845c10d3284d600caa5a86b819 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] alexeykudinkin commented on pull request #5052: [HUDI-3644] hoodie log scan bug cause data duplication bugfix

alexeykudinkin commented on pull request #5052: URL: https://github.com/apache/hudi/pull/5052#issuecomment-1070091573 @sekaiga can you please fill out the description properly? Is this PR still WIP? -- This is an automated message from the Apache Git Service. To respond to the message, p

[jira] [Assigned] (HUDI-3020) Create a utility to generate mainfest file

[ https://issues.apache.org/jira/browse/HUDI-3020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joyan Sil reassigned HUDI-3020: --- Assignee: Joyan Sil (was: Prashant Wason) > Create a utility to generate mainfest file > ---

[GitHub] [hudi] danny0405 commented on a change in pull request #3304: [HUDI-2198] Clean and reset the bootstrap events for coordinator when…

danny0405 commented on a change in pull request #3304: URL: https://github.com/apache/hudi/pull/3304#discussion_r828713714 ## File path: hudi-flink/src/main/java/org/apache/hudi/sink/partitioner/BucketAssignFunction.java ## @@ -194,9 +194,10 @@ private void processRecord(Hoodi

[GitHub] [hudi] hudi-bot removed a comment on pull request #4955: [HUDI-3549][Stacked on 4996][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot removed a comment on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1069998952 ## CI report: * d4b2b9aa43813e845c10d3284d600caa5a86b819 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4955: [HUDI-3549][Stacked on 4996][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot commented on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1070014635 ## CI report: * d4b2b9aa43813e845c10d3284d600caa5a86b819 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #4955: [HUDI-3549][Stacked on 4996][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot commented on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1069998952 ## CI report: * d4b2b9aa43813e845c10d3284d600caa5a86b819 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4955: [HUDI-3549][Stacked on 4996][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot removed a comment on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1069803516 ## CI report: * d4b2b9aa43813e845c10d3284d600caa5a86b819 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

hudi-bot removed a comment on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-1069800698 ## CI report: * 9ba5c351c32f9f364c30b4bc9a814075150d9728 UNKNOWN * 53b4cec7d05dd0bb5a622ffb33597471da7711de UNKNOWN * a19a778f36d2ad00357691d4a04a0da14f9b6

[GitHub] [hudi] hudi-bot commented on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

hudi-bot commented on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-1069910671 ## CI report: * 9ba5c351c32f9f364c30b4bc9a814075150d9728 UNKNOWN * 53b4cec7d05dd0bb5a622ffb33597471da7711de UNKNOWN * 438eecf51f0a2c08270ff6c1bca0ea2e4e83c594 Azur

[GitHub] [hudi] tjtoll edited a comment on issue #5047: [SUPPORT] Small file creation while writing to a Hudi Table

tjtoll edited a comment on issue #5047: URL: https://github.com/apache/hudi/issues/5047#issuecomment-1069788126 I am experiencing the same exact problem - also with AWS Glue. I tried the below settings and they increased the file sizes to a more appropriate size, but the insert is taking 1

[jira] [Updated] (HUDI-3643) Hive count throws exception when the table is empty and the path depth is less than 3

[ https://issues.apache.org/jira/browse/HUDI-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 董可伦 updated HUDI-3643: -- Description: Spark SQL create non-partition hudi table: {code:java} create table test_hudi_table (   id int,   name st

[jira] [Updated] (HUDI-3643) Hive count throws exception when the table is empty and the path depth is less than 3

[ https://issues.apache.org/jira/browse/HUDI-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 董可伦 updated HUDI-3643: -- Description: Spark SQL create non-partition hudi table: ```sql create table test_hudi_table (   id int,   name string,  

[jira] [Updated] (HUDI-3643) Hive count throws exception when the table is empty and the path depth is less than 3

[ https://issues.apache.org/jira/browse/HUDI-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 董可伦 updated HUDI-3643: -- Description: ## Spark SQL create non-partition hudi table: ```sql create table test_hudi_table (   id int,   name string

[GitHub] [hudi] hudi-bot commented on pull request #4974: [HUDI-3494] Consider triggering condition of MOR compaction during archival

hudi-bot commented on pull request #4974: URL: https://github.com/apache/hudi/pull/4974#issuecomment-1069822816 ## CI report: * 681207d07b3297c25fa32a02d6570dc9110c8aa7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4974: [HUDI-3494] Consider triggering condition of MOR compaction during archival

hudi-bot removed a comment on pull request #4974: URL: https://github.com/apache/hudi/pull/4974#issuecomment-1069779096 ## CI report: * 8f7ed79ebab7d10624e7e5e9a641cc41e802ef98 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] prashantwason commented on pull request #4274: [HUDI-2974] Make the prefix for metrics name configurable

prashantwason commented on pull request #4274: URL: https://github.com/apache/hudi/pull/4274#issuecomment-1069818077 @yihua @nsivabalan I think this commit broke metrics for usage like this: HoodieMetricsConfig.Builder metricsConfig = HoodieMetricsConfig.newBuilder()

[GitHub] [hudi] yihua commented on a change in pull request #4962: [HUDI-3355] Issue with out of order commits in the timeline when ingestion writers using SparkAllowUpdateStrategy

yihua commented on a change in pull request #4962: URL: https://github.com/apache/hudi/pull/4962#discussion_r828623933 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java ## @@ -120,6 +121,7 @@ protected transient Asy

[hudi] branch master updated (91849c3 -> 8ca9a54)

This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 91849c3 [HUDI-3607] Support backend switch in HoodieFlinkStreamer (#5032) add 8ca9a54 [Hudi-3376] Add an option

[GitHub] [hudi] yihua merged pull request #4994: [Hudi-3376] Add an option to skip under deletion files for HoodieMetadataTableValidator

yihua merged pull request #4994: URL: https://github.com/apache/hudi/pull/4994 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[GitHub] [hudi] hudi-bot removed a comment on pull request #4975: [HUDI-3404] Automatically adjust write configs based on metadata table and write concurrency mode

hudi-bot removed a comment on pull request #4975: URL: https://github.com/apache/hudi/pull/4975#issuecomment-1069770565 ## CI report: * 0c1aad3b608f4eea6d5882b27e819ee138dd3748 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4975: [HUDI-3404] Automatically adjust write configs based on metadata table and write concurrency mode

hudi-bot commented on pull request #4975: URL: https://github.com/apache/hudi/pull/4975#issuecomment-1069813192 ## CI report: * 1bc5f116d28f5ef2f6ea522a71688ea49c04b693 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] kywe665 edited a comment on pull request #5055: [MINOR] - blog css adj and adding images

kywe665 edited a comment on pull request #5055: URL: https://github.com/apache/hudi/pull/5055#issuecomment-1069810983 I will do pages 2 & 3 soon, but wanted to push this one out first. Next steps: 1. I will add author and source 2. I will merge talks and articles in 3. I will appl

[GitHub] [hudi] kywe665 commented on pull request #5055: [MINOR] - blog css adj and adding images

kywe665 commented on pull request #5055: URL: https://github.com/apache/hudi/pull/5055#issuecomment-1069810983 I will do pages 2 & 3 soon, but wanted to push this one out first. Next I will add Author and Source -- This is an automated message from the Apache Git Service. To respond to t

[GitHub] [hudi] yihua commented on a change in pull request #4994: [Hudi-3376] Add an option to skip under deletion files for HoodieMetadataTableValidator

yihua commented on a change in pull request #4994: URL: https://github.com/apache/hudi/pull/4994#discussion_r828610748 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java ## @@ -345,16 +358,59 @@ public void doMetadataTableVal

[GitHub] [hudi] kywe665 commented on pull request #5055: [MINOR] - blog css adj and adding images

kywe665 commented on pull request #5055: URL: https://github.com/apache/hudi/pull/5055#issuecomment-1069810122  -- This is an automated message from the Apache Git Service. T

[GitHub] [hudi] hudi-bot commented on pull request #4955: [HUDI-3549][WIP][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot commented on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1069803516 ## CI report: * d4b2b9aa43813e845c10d3284d600caa5a86b819 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4955: [HUDI-3549][WIP][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot removed a comment on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1069800677 ## CI report: * 846b0d8949a8178d5d203970fa4baffb5a1163d8 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[jira] [Updated] (HUDI-3650) Revisit all usages of filterPendingCompactionTimeline()

[ https://issues.apache.org/jira/browse/HUDI-3650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3650: Fix Version/s: 0.11.0 > Revisit all usages of filterPendingCompactionTimeline() > -

[jira] [Updated] (HUDI-3650) Revisit all usages of filterPendingCompactionTimeline()

[ https://issues.apache.org/jira/browse/HUDI-3650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3650: Affects Version/s: (was: 0.11.0) > Revisit all usages of filterPendingCompactionTimeline() > --

[GitHub] [hudi] yihua commented on a change in pull request #4172: [HUDI-2892][BUG]Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

yihua commented on a change in pull request #4172: URL: https://github.com/apache/hudi/pull/4172#discussion_r828592002 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTimelineArchiveLog.java ## @@ -158,8 +158,12 @@ public boolean archiveI

[jira] [Created] (HUDI-3650) Revisit all usages of filterPendingCompactionTimeline()

Ethan Guo created HUDI-3650: --- Summary: Revisit all usages of filterPendingCompactionTimeline() Key: HUDI-3650 URL: https://issues.apache.org/jira/browse/HUDI-3650 Project: Apache Hudi Issue Type:

[GitHub] [hudi] hudi-bot commented on pull request #4955: [HUDI-3549][WIP][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot commented on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1069800677 ## CI report: * 846b0d8949a8178d5d203970fa4baffb5a1163d8 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #4955: [HUDI-3549][WIP][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot removed a comment on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1069799054 ## CI report: * 088c5bbb80c355ff88d72b8efa5618c4d3118dee Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot removed a comment on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

hudi-bot removed a comment on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-1069799095 ## CI report: * 9ba5c351c32f9f364c30b4bc9a814075150d9728 UNKNOWN * 53b4cec7d05dd0bb5a622ffb33597471da7711de UNKNOWN * a19a778f36d2ad00357691d4a04a0da14f9b6

[GitHub] [hudi] hudi-bot commented on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

hudi-bot commented on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-1069800698 ## CI report: * 9ba5c351c32f9f364c30b4bc9a814075150d9728 UNKNOWN * 53b4cec7d05dd0bb5a622ffb33597471da7711de UNKNOWN * a19a778f36d2ad00357691d4a04a0da14f9b624f Azur

[GitHub] [hudi] hudi-bot commented on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

hudi-bot commented on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-1069799095 ## CI report: * 9ba5c351c32f9f364c30b4bc9a814075150d9728 UNKNOWN * 53b4cec7d05dd0bb5a622ffb33597471da7711de UNKNOWN * a19a778f36d2ad00357691d4a04a0da14f9b624f Azur

[GitHub] [hudi] hudi-bot removed a comment on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

hudi-bot removed a comment on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-1069322997 ## CI report: * 9ba5c351c32f9f364c30b4bc9a814075150d9728 UNKNOWN * 53b4cec7d05dd0bb5a622ffb33597471da7711de UNKNOWN * a19a778f36d2ad00357691d4a04a0da14f9b6

[GitHub] [hudi] hudi-bot removed a comment on pull request #4955: [HUDI-3549][WIP][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot removed a comment on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1069796629 ## CI report: * ec422719c01c7d5d4418276f18e7be3022faac5b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4955: [HUDI-3549][WIP][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot commented on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1069799054 ## CI report: * 088c5bbb80c355ff88d72b8efa5618c4d3118dee Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot commented on pull request #4955: [HUDI-3549][WIP][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot commented on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1069796629 ## CI report: * ec422719c01c7d5d4418276f18e7be3022faac5b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4955: [HUDI-3549][WIP][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot removed a comment on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1069794614 ## CI report: * ec422719c01c7d5d4418276f18e7be3022faac5b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4955: [HUDI-3549][WIP][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot removed a comment on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1069792521 ## CI report: * ec422719c01c7d5d4418276f18e7be3022faac5b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4955: [HUDI-3549][WIP][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot commented on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1069794614 ## CI report: * ec422719c01c7d5d4418276f18e7be3022faac5b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4955: [HUDI-3549][WIP][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot removed a comment on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1061216142 ## CI report: * ec422719c01c7d5d4418276f18e7be3022faac5b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4955: [HUDI-3549][WIP][DO_NOT_MERGE] Removing "spark-avro" bundling from Hudi by default

hudi-bot commented on pull request #4955: URL: https://github.com/apache/hudi/pull/4955#issuecomment-1069792521 ## CI report: * ec422719c01c7d5d4418276f18e7be3022faac5b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] tjtoll commented on issue #5047: [SUPPORT] Small file creation while writing to a Hudi Table

tjtoll commented on issue #5047: URL: https://github.com/apache/hudi/issues/5047#issuecomment-1069788126 I am experiencing the same exact problem. I tried the below settings and they increased the file sizes to a more appropriate size, but the insert is taking 10x longer. Also interested i

[GitHub] [hudi] yihua commented on a change in pull request #4856: [HUDI-2439] Replace RDD with HoodieData in HoodieSparkTable and commit executors

yihua commented on a change in pull request #4856: URL: https://github.com/apache/hudi/pull/4856#discussion_r828562205 ## File path: hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/table/action/commit/TestDeleteHelper.java ## @@ -1,194 +0,0 @@ -/* - * Licensed to t

[GitHub] [hudi] tisonkun commented on a change in pull request #3304: [HUDI-2198] Clean and reset the bootstrap events for coordinator when…

tisonkun commented on a change in pull request #3304: URL: https://github.com/apache/hudi/pull/3304#discussion_r828561997 ## File path: hudi-flink/src/main/java/org/apache/hudi/sink/partitioner/BucketAssignFunction.java ## @@ -194,9 +194,10 @@ private void processRecord(Hoodie

[GitHub] [hudi] yihua commented on a change in pull request #4856: [HUDI-2439] Replace RDD with HoodieData in HoodieSparkTable and commit executors

yihua commented on a change in pull request #4856: URL: https://github.com/apache/hudi/pull/4856#discussion_r828561373 ## File path: hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/client/HoodieFlinkWriteClient.java ## @@ -363,7 +365,7 @@ public void completeCompac

[GitHub] [hudi] kywe665 opened a new pull request #5055: [MINOR] - blog css adj and adding images

kywe665 opened a new pull request #5055: URL: https://github.com/apache/hudi/pull/5055 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpose

[jira] [Updated] (HUDI-3610) Validate Hudi Kafka Connect Sink writing to S3

[ https://issues.apache.org/jira/browse/HUDI-3610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3610: - Sprint: Cont' improve - 2022/03/7 > Validate Hudi Kafka Connect Sink writing to S3 > -

[jira] [Updated] (HUDI-3610) Validate Hudi Kafka Connect Sink writing to S3

[ https://issues.apache.org/jira/browse/HUDI-3610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3610: - Component/s: kafka-connect > Validate Hudi Kafka Connect Sink writing to S3 >

[jira] [Updated] (HUDI-3579) Add timeline commands in hudi-cli

[ https://issues.apache.org/jira/browse/HUDI-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3579: - Sprint: Cont' improve - 2022/03/7 > Add timeline commands in hudi-cli > -

[GitHub] [hudi] hudi-bot commented on pull request #4974: [HUDI-3494] Consider triggering condition of MOR compaction during archival

hudi-bot commented on pull request #4974: URL: https://github.com/apache/hudi/pull/4974#issuecomment-1069779096 ## CI report: * 8f7ed79ebab7d10624e7e5e9a641cc41e802ef98 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4974: [HUDI-3494] Consider triggering condition of MOR compaction during archival

hudi-bot removed a comment on pull request #4974: URL: https://github.com/apache/hudi/pull/4974#issuecomment-1069777232 ## CI report: * 8f7ed79ebab7d10624e7e5e9a641cc41e802ef98 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4974: [HUDI-3494] Consider triggering condition of MOR compaction during archival

hudi-bot commented on pull request #4974: URL: https://github.com/apache/hudi/pull/4974#issuecomment-1069777232 ## CI report: * 8f7ed79ebab7d10624e7e5e9a641cc41e802ef98 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4974: [HUDI-3494] Consider triggering condition of MOR compaction during archival

hudi-bot removed a comment on pull request #4974: URL: https://github.com/apache/hudi/pull/4974#issuecomment-1061374163 ## CI report: * 8f7ed79ebab7d10624e7e5e9a641cc41e802ef98 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4286: [WIP][HUDI-2955] Upgrade Hadoop to 3.3.1, Hive to 3.1.2, HBase to 2.4.8

hudi-bot commented on pull request #4286: URL: https://github.com/apache/hudi/pull/4286#issuecomment-1069776883 ## CI report: * c2738a5a1cd6f5ea8cffa966d7dc332f2d211f29 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4286: [WIP][HUDI-2955] Upgrade Hadoop to 3.3.1, Hive to 3.1.2, HBase to 2.4.8

hudi-bot removed a comment on pull request #4286: URL: https://github.com/apache/hudi/pull/4286#issuecomment-1069734507 ## CI report: * dbe231115001abd287e8d948d51011b708c02f7c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] yihua commented on a change in pull request #4974: [HUDI-3494] Consider triggering condition of MOR compaction during archival

yihua commented on a change in pull request #4974: URL: https://github.com/apache/hudi/pull/4974#discussion_r828554793 ## File path: hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/io/TestHoodieTimelineArchiver.java ## @@ -946,6 +961,152 @@ public void testArchive

[GitHub] [hudi] yihua commented on a change in pull request #4974: [HUDI-3494] Consider triggering condition of MOR compaction during archival

yihua commented on a change in pull request #4974: URL: https://github.com/apache/hudi/pull/4974#discussion_r828554297 ## File path: hudi-common/src/main/java/org/apache/hudi/common/util/CompactionUtils.java ## @@ -195,10 +196,76 @@ public static HoodieCompactionPlan getCompa

[GitHub] [hudi] hudi-bot removed a comment on pull request #4975: [HUDI-3404] Automatically adjust write configs based on metadata table and write concurrency mode

hudi-bot removed a comment on pull request #4975: URL: https://github.com/apache/hudi/pull/4975#issuecomment-1069768813 ## CI report: * 0c1aad3b608f4eea6d5882b27e819ee138dd3748 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4975: [HUDI-3404] Automatically adjust write configs based on metadata table and write concurrency mode

hudi-bot commented on pull request #4975: URL: https://github.com/apache/hudi/pull/4975#issuecomment-1069770565 ## CI report: * 0c1aad3b608f4eea6d5882b27e819ee138dd3748 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] yihua commented on a change in pull request #4974: [HUDI-3494] Consider triggering condition of MOR compaction during archival

yihua commented on a change in pull request #4974: URL: https://github.com/apache/hudi/pull/4974#discussion_r828553673 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/compact/ScheduleCompactionActionExecutor.java ## @@ -128,27 +129,25 @@

[GitHub] [hudi] hudi-bot removed a comment on pull request #4975: [HUDI-3404] Automatically adjust write configs based on metadata table and write concurrency mode

hudi-bot removed a comment on pull request #4975: URL: https://github.com/apache/hudi/pull/4975#issuecomment-1061386882 ## CI report: * 0c1aad3b608f4eea6d5882b27e819ee138dd3748 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4975: [HUDI-3404] Automatically adjust write configs based on metadata table and write concurrency mode

hudi-bot commented on pull request #4975: URL: https://github.com/apache/hudi/pull/4975#issuecomment-1069768813 ## CI report: * 0c1aad3b608f4eea6d5882b27e819ee138dd3748 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[jira] [Updated] (HUDI-3649) Add HoodieTableConfig defaults to HoodieWriteConfig

[ https://issues.apache.org/jira/browse/HUDI-3649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3649: Priority: Major (was: Blocker) > Add HoodieTableConfig defaults to HoodieWriteConfig >

[GitHub] [hudi] yihua commented on pull request #4975: [HUDI-3404] Automatically adjust write configs based on metadata table and write concurrency mode

yihua commented on pull request #4975: URL: https://github.com/apache/hudi/pull/4975#issuecomment-1069762836 > based on our discussion, lets add table type check and check for compaction only for MOR. bcoz, incase of Deltastreamer, no explicit compaction configs are set, but still we need

[GitHub] [hudi] yihua commented on a change in pull request #4975: [HUDI-3404] Automatically adjust write configs based on metadata table and write concurrency mode

yihua commented on a change in pull request #4975: URL: https://github.com/apache/hudi/pull/4975#discussion_r828548624 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java ## @@ -2375,19 +2375,49 @@ protected void setDefaults

[jira] [Updated] (HUDI-3649) Add HoodieTableConfig defaults to HoodieWriteConfig

[ https://issues.apache.org/jira/browse/HUDI-3649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3649: Description: HoodieWriteConfig does not set defaults from HoodieTableConfig.  We need to see if some HoodieT

[jira] [Updated] (HUDI-3649) Add HoodieTableConfig defaults to HoodieWriteConfig

[ https://issues.apache.org/jira/browse/HUDI-3649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3649: Description: HoodieWriteConfig does not set defaults from HoodieWriteConfig.  We need to see if some config

[jira] [Updated] (HUDI-3649) Add HoodieTableConfig defaults to HoodieWriteConfig

[ https://issues.apache.org/jira/browse/HUDI-3649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3649: Description: HoodieWriteConfig does not set defaults from HoodieWriteConfig.  W  (was: HoodieWriteConfig do

  1   2   3   >