[GitHub] [hudi] hudi-bot commented on pull request #4167: [HUDI-2880] Fixing loading of props from default dir

hudi-bot commented on pull request #4167: URL: https://github.com/apache/hudi/pull/4167#issuecomment-983379782 ## CI report: * 8c008662ef556f9333d6512ad31d09837e2b57cb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] dik111 commented on issue #4030: [SUPPORT] Flink uses updated fields to update data

dik111 commented on issue #4030: URL: https://github.com/apache/hudi/issues/4030#issuecomment-983379536 > Hi, @dik111 I want to sure with you, is your use case occur a schema evolution in updated changelog or schema in fixed, only lack of some fields in updated changelog? Serializing the `

[GitHub] [hudi] hudi-bot removed a comment on pull request #4167: [HUDI-2880] Fixing loading of props from default dir

hudi-bot removed a comment on pull request #4167: URL: https://github.com/apache/hudi/pull/4167#issuecomment-983327655 ## CI report: * 8c008662ef556f9333d6512ad31d09837e2b57cb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4175: [HUDI-2883] Refactor hive sync tool / config to use reflection and standardize configs

hudi-bot commented on pull request #4175: URL: https://github.com/apache/hudi/pull/4175#issuecomment-983365431 ## CI report: * c1738f4df051621cc349f95457f5e65dfe276cec Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4175: [HUDI-2883] Refactor hive sync tool / config to use reflection and standardize configs

hudi-bot removed a comment on pull request #4175: URL: https://github.com/apache/hudi/pull/4175#issuecomment-983361033 ## CI report: * c1738f4df051621cc349f95457f5e65dfe276cec UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4175: [HUDI-2883] [HUDI-2883] Refactor hive sync tool / config to use reflection and standardize configs

hudi-bot commented on pull request #4175: URL: https://github.com/apache/hudi/pull/4175#issuecomment-983361033 ## CI report: * c1738f4df051621cc349f95457f5e65dfe276cec UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[jira] [Updated] (HUDI-2883) Refactor Hive Sync tool /config to use reflection and move to hudi sync common package

[ https://issues.apache.org/jira/browse/HUDI-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2883: - Labels: pull-request-available (was: ) > Refactor Hive Sync tool /config to use reflection and mo

[GitHub] [hudi] rmahindra123 opened a new pull request #4175: [HUDI-2883] [HUDI-2883] Refactor hive sync tool / config to use reflection and standardize configs

rmahindra123 opened a new pull request #4175: URL: https://github.com/apache/hudi/pull/4175 Refactor hive sync tool / config to use reflection and standardize configs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[GitHub] [hudi] hudi-bot removed a comment on pull request #4172: [HUDI-2892][BUG]Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

hudi-bot removed a comment on pull request #4172: URL: https://github.com/apache/hudi/pull/4172#issuecomment-983318132 ## CI report: * bb29721e619d7e04e926613d8b7992db95eb3306 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4172: [HUDI-2892][BUG]Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

hudi-bot commented on pull request #4172: URL: https://github.com/apache/hudi/pull/4172#issuecomment-983350101 ## CI report: * bb29721e619d7e04e926613d8b7992db95eb3306 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[jira] [Commented] (HUDI-2781) Test 0.10 RC for Spark 3.x

[ https://issues.apache.org/jira/browse/HUDI-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17451546#comment-17451546 ] Raymond Xu commented on HUDI-2781: -- [~biyan900...@gmail.com] a quick discussion recap: #

[GitHub] [hudi] hudi-bot commented on pull request #4173: [MINOR] [WIP] Check on TestHBaseIndex

hudi-bot commented on pull request #4173: URL: https://github.com/apache/hudi/pull/4173#issuecomment-983336858 ## CI report: * e44ac377bd15d4e66bbf641ec5bf18253dadc01b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4173: [MINOR] [WIP] Check on TestHBaseIndex

hudi-bot removed a comment on pull request #4173: URL: https://github.com/apache/hudi/pull/4173#issuecomment-983307961 ## CI report: * e44ac377bd15d4e66bbf641ec5bf18253dadc01b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] xiarixiaoyao commented on pull request #3808: [HUDI-2560] introduce id_based schema to support full schema evolution.

xiarixiaoyao commented on pull request #3808: URL: https://github.com/apache/hudi/pull/3808#issuecomment-983334334 @bvaradar already rebase the code and addressed all comments. could you pls help me review those code again. thanks -- This is an automated message from the Apache Git S

[GitHub] [hudi] esaeki opened a new issue #4174: cannot delete data MOR mode

esaeki opened a new issue #4174: URL: https://github.com/apache/hudi/issues/4174 I would like to delete some data in S3 by using Glue and Hudi, but encountered the following error. ``` An error occurred while calling 0128.save. Illegal character in: tableName_record ```

[GitHub] [hudi] xiarixiaoyao commented on issue #4135: [SUPPORT] Zordering clustering on a moderate size dataset taking large amounts of time.

xiarixiaoyao commented on issue #4135: URL: https://github.com/apache/hudi/issues/4135#issuecomment-983331732 @rubenssoto two ways 1）：spark.sql("set hoodie.enable.data.skipping = True") 2）：df = spark.read.format("hudi").option("hoodie.enable.data.skipping", "True").load("/tmp/de

[GitHub] [hudi] hudi-bot commented on pull request #4167: [HUDI-2880] Fixing loading of props from default dir

hudi-bot commented on pull request #4167: URL: https://github.com/apache/hudi/pull/4167#issuecomment-983327655 ## CI report: * 8c008662ef556f9333d6512ad31d09837e2b57cb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4167: [HUDI-2880] Fixing loading of props from default dir

hudi-bot removed a comment on pull request #4167: URL: https://github.com/apache/hudi/pull/4167#issuecomment-983292160 ## CI report: * 8c008662ef556f9333d6512ad31d09837e2b57cb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] danny0405 commented on pull request #4167: [HUDI-2880] Fixing loading of props from default dir

danny0405 commented on pull request #4167: URL: https://github.com/apache/hudi/pull/4167#issuecomment-983326804 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[jira] [Resolved] (HUDI-2891) Kafka-connect sink still uses timeline-server-based markers as default

[ https://issues.apache.org/jira/browse/HUDI-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-2891. -- > Kafka-connect sink still uses timeline-server-based markers as default > -

[jira] [Commented] (HUDI-2891) Kafka-connect sink still uses timeline-server-based markers as default

[ https://issues.apache.org/jira/browse/HUDI-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17451521#comment-17451521 ] Danny Chen commented on HUDI-2891: -- Fixed via master branch: ea009b55a385564f76beb99468f3

[jira] [Resolved] (HUDI-2898) `AvroRuntimeException` thrown when clustering payloads w/ no "ts" field

[ https://issues.apache.org/jira/browse/HUDI-2898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-2898. -- > `AvroRuntimeException` thrown when clustering payloads w/ no "ts" field >

[jira] [Commented] (HUDI-2898) `AvroRuntimeException` thrown when clustering payloads w/ no "ts" field

[ https://issues.apache.org/jira/browse/HUDI-2898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17451520#comment-17451520 ] Danny Chen commented on HUDI-2898: -- Fixed via master branch: 24380c20606d63c7c129cb45a73f

[hudi] branch master updated (24380c2 -> 9b254b6)

This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 24380c2 Revert "[HUDI-2855] Change the default value of 'PAYLOAD_CLASS_NAME' to 'DefaultHoodieRecordPayload' (#411

[GitHub] [hudi] danny0405 merged pull request #4171: Revert "[HUDI-2856] Bit cask disk map delete modified"

danny0405 merged pull request #4171: URL: https://github.com/apache/hudi/pull/4171 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[GitHub] [hudi] danny0405 commented on pull request #4171: Revert "[HUDI-2856] Bit cask disk map delete modified"

danny0405 commented on pull request #4171: URL: https://github.com/apache/hudi/pull/4171#issuecomment-983318814 > @danny0405 : Can we go ahead and merge this in. Or do you have any clarifications. We can, would merge it. -- This is an automated message from the Apache Git Service.

[GitHub] [hudi] hudi-bot commented on pull request #4172: [HUDI-2892][BUG]Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

hudi-bot commented on pull request #4172: URL: https://github.com/apache/hudi/pull/4172#issuecomment-983318132 ## CI report: * bb29721e619d7e04e926613d8b7992db95eb3306 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4172: [HUDI-2892][BUG]Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

hudi-bot removed a comment on pull request #4172: URL: https://github.com/apache/hudi/pull/4172#issuecomment-983316262 ## CI report: * bb29721e619d7e04e926613d8b7992db95eb3306 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] zhangyue19921010 commented on pull request #4172: [HUDI-2892][BUG]Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

zhangyue19921010 commented on pull request #4172: URL: https://github.com/apache/hudi/pull/4172#issuecomment-983316754 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [hudi] hudi-bot removed a comment on pull request #4172: [HUDI-2892][BUG]Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

hudi-bot removed a comment on pull request #4172: URL: https://github.com/apache/hudi/pull/4172#issuecomment-983299157 ## CI report: * 365c5828cd269839984fc849386ecafee0a31d1d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4172: [HUDI-2892][BUG]Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

hudi-bot commented on pull request #4172: URL: https://github.com/apache/hudi/pull/4172#issuecomment-983316262 ## CI report: * bb29721e619d7e04e926613d8b7992db95eb3306 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot commented on pull request #4173: [MINOR] [WIP] Check on TestHBaseIndex

hudi-bot commented on pull request #4173: URL: https://github.com/apache/hudi/pull/4173#issuecomment-983307961 ## CI report: * e44ac377bd15d4e66bbf641ec5bf18253dadc01b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4173: [MINOR] [WIP] Check on TestHBaseIndex

hudi-bot removed a comment on pull request #4173: URL: https://github.com/apache/hudi/pull/4173#issuecomment-983306932 ## CI report: * e44ac377bd15d4e66bbf641ec5bf18253dadc01b UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4173: [MINOR] [WIP] Check on TestHBaseIndex

hudi-bot commented on pull request #4173: URL: https://github.com/apache/hudi/pull/4173#issuecomment-983306932 ## CI report: * e44ac377bd15d4e66bbf641ec5bf18253dadc01b UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[GitHub] [hudi] xushiyan opened a new pull request #4173: [MINOR] [WIP] Check on TestHBaseIndex

xushiyan opened a new pull request #4173: URL: https://github.com/apache/hudi/pull/4173 ## Committer checklist - [ ] Has a corresponding JIRA in PR title & commit - [ ] Commit message is descriptive of the change - [ ] CI is green - [ ] Necessary doc cha

[GitHub] [hudi] hudi-bot removed a comment on pull request #4172: [HUDI-2892][BUG]Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

hudi-bot removed a comment on pull request #4172: URL: https://github.com/apache/hudi/pull/4172#issuecomment-983298059 ## CI report: * 365c5828cd269839984fc849386ecafee0a31d1d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4172: [HUDI-2892][BUG]Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

hudi-bot commented on pull request #4172: URL: https://github.com/apache/hudi/pull/4172#issuecomment-983299157 ## CI report: * 365c5828cd269839984fc849386ecafee0a31d1d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] YannByron commented on pull request #4085: [WIP][HUDI-2781] Test 0.10 RC for Spark3.x

YannByron commented on pull request #4085: URL: https://github.com/apache/hudi/pull/4085#issuecomment-983298846 related to #4138 . close this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] YannByron closed pull request #4085: [WIP][HUDI-2781] Test 0.10 RC for Spark3.x

YannByron closed pull request #4085: URL: https://github.com/apache/hudi/pull/4085 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[GitHub] [hudi] hudi-bot commented on pull request #4172: [HUDI-2892][BUG]Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

hudi-bot commented on pull request #4172: URL: https://github.com/apache/hudi/pull/4172#issuecomment-983298059 ## CI report: * 365c5828cd269839984fc849386ecafee0a31d1d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4172: [HUDI-2892][BUG]Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

hudi-bot removed a comment on pull request #4172: URL: https://github.com/apache/hudi/pull/4172#issuecomment-983249090 ## CI report: * 365c5828cd269839984fc849386ecafee0a31d1d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] zhangyue19921010 commented on pull request #3897: [HUDI-2658] When disable auto clean, do not check if MIN_COMMITS_TO_KEEP was larger CLEANER_COMMITS_RETAINED or not.

zhangyue19921010 commented on pull request #3897: URL: https://github.com/apache/hudi/pull/3897#issuecomment-983298086 Just fix conflict from master @xushiyan PTAL :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[GitHub] [hudi] rubenssoto commented on issue #4135: [SUPPORT] Zordering clustering on a moderate size dataset taking large amounts of time.

rubenssoto commented on issue #4135: URL: https://github.com/apache/hudi/issues/4135#issuecomment-98329 Thank you @xiarixiaoyao I will test with bigger values on hoodie.clustering.plan.strategy.max.bytes.per.group. Only one more question, I want to demonstrate data skipping t

[GitHub] [hudi] nsivabalan commented on pull request #4171: Revert "[HUDI-2856] Bit cask disk map delete modified"

nsivabalan commented on pull request #4171: URL: https://github.com/apache/hudi/pull/4171#issuecomment-983292162 @danny0405 : Can we go ahead and merge this in. Or do you have any clarifications. -- This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [hudi] hudi-bot removed a comment on pull request #4167: [HUDI-2880] Fixing loading of props from default dir

hudi-bot removed a comment on pull request #4167: URL: https://github.com/apache/hudi/pull/4167#issuecomment-983262850 ## CI report: * 1c3bad0199b6744fecc1f3638ea30aac788b35bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4167: [HUDI-2880] Fixing loading of props from default dir

hudi-bot commented on pull request #4167: URL: https://github.com/apache/hudi/pull/4167#issuecomment-983292160 ## CI report: * 8c008662ef556f9333d6512ad31d09837e2b57cb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot commented on pull request #3897: [HUDI-2658] When disable auto clean, do not check if MIN_COMMITS_TO_KEEP was larger CLEANER_COMMITS_RETAINED or not.

hudi-bot commented on pull request #3897: URL: https://github.com/apache/hudi/pull/3897#issuecomment-983281099 ## CI report: * 3025fe43776e63a4d96cddd0357e1f0ec76eff35 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #3897: [HUDI-2658] When disable auto clean, do not check if MIN_COMMITS_TO_KEEP was larger CLEANER_COMMITS_RETAINED or not.

hudi-bot removed a comment on pull request #3897: URL: https://github.com/apache/hudi/pull/3897#issuecomment-983251919 ## CI report: * 1e6c7fedbef87cab4843eab4831202db437118e0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4167: [HUDI-2880] Fixing loading of props from default dir

hudi-bot commented on pull request #4167: URL: https://github.com/apache/hudi/pull/4167#issuecomment-983262850 ## CI report: * 1c3bad0199b6744fecc1f3638ea30aac788b35bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4167: [HUDI-2880] Fixing loading of props from default dir

hudi-bot removed a comment on pull request #4167: URL: https://github.com/apache/hudi/pull/4167#issuecomment-983258428 ## CI report: * 1c3bad0199b6744fecc1f3638ea30aac788b35bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] zhangyue19921010 commented on pull request #3887: [HUDI-2648] Retry FileSystem action instead of failed directly.

zhangyue19921010 commented on pull request #3887: URL: https://github.com/apache/hudi/pull/3887#issuecomment-983260291 > cos(Cloud Object Storage) need this patch help to retry https://cloud.tencent.com/product/cos Thanks for @mincwang 's info, It looks like https://intl.cloud.tence

[GitHub] [hudi] hudi-bot commented on pull request #4167: [HUDI-2880] Fixing loading of props from default dir

hudi-bot commented on pull request #4167: URL: https://github.com/apache/hudi/pull/4167#issuecomment-983258428 ## CI report: * 1c3bad0199b6744fecc1f3638ea30aac788b35bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4167: [HUDI-2880] Fixing loading of props from default dir

hudi-bot removed a comment on pull request #4167: URL: https://github.com/apache/hudi/pull/4167#issuecomment-983221885 ## CI report: * 1c3bad0199b6744fecc1f3638ea30aac788b35bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] xiarixiaoyao commented on issue #4135: [SUPPORT] Zordering clustering on a moderate size dataset taking large amounts of time.

xiarixiaoyao commented on issue #4135: URL: https://github.com/apache/hudi/issues/4135#issuecomment-983254085 @rubenssoto @vinothchandar i have run the test。 suggestions： if you use z-order/hilbert in clustering, pls set option("hoodie.clustering.plan.strategy.max.bytes.per.group", *

[GitHub] [hudi] hudi-bot commented on pull request #3897: [HUDI-2658] When disable auto clean, do not check if MIN_COMMITS_TO_KEEP was larger CLEANER_COMMITS_RETAINED or not.

hudi-bot commented on pull request #3897: URL: https://github.com/apache/hudi/pull/3897#issuecomment-983251919 ## CI report: * 1e6c7fedbef87cab4843eab4831202db437118e0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #3897: [HUDI-2658] When disable auto clean, do not check if MIN_COMMITS_TO_KEEP was larger CLEANER_COMMITS_RETAINED or not.

hudi-bot removed a comment on pull request #3897: URL: https://github.com/apache/hudi/pull/3897#issuecomment-983250066 ## CI report: * 1e6c7fedbef87cab4843eab4831202db437118e0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot removed a comment on pull request #3897: [HUDI-2658] When disable auto clean, do not check if MIN_COMMITS_TO_KEEP was larger CLEANER_COMMITS_RETAINED or not.

hudi-bot removed a comment on pull request #3897: URL: https://github.com/apache/hudi/pull/3897#issuecomment-961588774 ## CI report: * 1e6c7fedbef87cab4843eab4831202db437118e0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #3897: [HUDI-2658] When disable auto clean, do not check if MIN_COMMITS_TO_KEEP was larger CLEANER_COMMITS_RETAINED or not.

hudi-bot commented on pull request #3897: URL: https://github.com/apache/hudi/pull/3897#issuecomment-983250066 ## CI report: * 1e6c7fedbef87cab4843eab4831202db437118e0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4172: [HUDI-2892][BUG]Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

hudi-bot removed a comment on pull request #4172: URL: https://github.com/apache/hudi/pull/4172#issuecomment-983247981 ## CI report: * 365c5828cd269839984fc849386ecafee0a31d1d UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4172: [HUDI-2892][BUG]Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

hudi-bot commented on pull request #4172: URL: https://github.com/apache/hudi/pull/4172#issuecomment-983249090 ## CI report: * 365c5828cd269839984fc849386ecafee0a31d1d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #4172: [HUDI-2892][BUG]Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

hudi-bot commented on pull request #4172: URL: https://github.com/apache/hudi/pull/4172#issuecomment-983247981 ## CI report: * 365c5828cd269839984fc849386ecafee0a31d1d UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[GitHub] [hudi] yihua commented on pull request #4171: Revert "[HUDI-2856] Bit cask disk map delete modified"

yihua commented on pull request #4171: URL: https://github.com/apache/hudi/pull/4171#issuecomment-983247432 > Can we give some explanation why this exception throws with the #4116, the code logic does not change ? The only difference is that the new code may throw exception in the `finally

[GitHub] [hudi] zhangyue19921010 opened a new pull request #4172: [HUDI-2892][BUG]Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

zhangyue19921010 opened a new pull request #4172: URL: https://github.com/apache/hudi/pull/4172 details are in https://github.com/apache/hudi/issues/4163 https://issues.apache.org/jira/browse/HUDI-2892 ## What is the purpose of the pull request If there's a pending clusterin

[GitHub] [hudi] zhangyue19921010 closed pull request #4164: [HUDI-2892][BUG]Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

zhangyue19921010 closed pull request #4164: URL: https://github.com/apache/hudi/pull/4164 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-

[GitHub] [hudi] hudi-bot removed a comment on pull request #4164: [HUDI-2892][BUG]Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

hudi-bot removed a comment on pull request #4164: URL: https://github.com/apache/hudi/pull/4164#issuecomment-983105480 ## CI report: * 365c5828cd269839984fc849386ecafee0a31d1d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4164: [HUDI-2892][BUG]Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

hudi-bot commented on pull request #4164: URL: https://github.com/apache/hudi/pull/4164#issuecomment-983245723 ## CI report: * 365c5828cd269839984fc849386ecafee0a31d1d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] zhangyue19921010 commented on pull request #4164: [HUDI-2892][BUG]Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

zhangyue19921010 commented on pull request #4164: URL: https://github.com/apache/hudi/pull/4164#issuecomment-983245018 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [hudi] zhangyue19921010 removed a comment on pull request #4164: [HUDI-2892][BUG]Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

zhangyue19921010 removed a comment on pull request #4164: URL: https://github.com/apache/hudi/pull/4164#issuecomment-983095842 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [hudi] yihua commented on a change in pull request #4167: [HUDI-2880] Fixing loading of props from default dir

yihua commented on a change in pull request #4167: URL: https://github.com/apache/hudi/pull/4167#discussion_r759819354 ## File path: hudi-common/src/main/java/org/apache/hudi/common/config/DFSPropertiesConfiguration.java ## @@ -97,11 +99,7 @@ public static TypedProperties load

[GitHub] [hudi] hudi-bot commented on pull request #4171: Revert "[HUDI-2856] Bit cask disk map delete modified"

hudi-bot commented on pull request #4171: URL: https://github.com/apache/hudi/pull/4171#issuecomment-983232403 ## CI report: * 7c8ddb42aa8e8ae597c00635c415368860604ead Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4171: Revert "[HUDI-2856] Bit cask disk map delete modified"

hudi-bot removed a comment on pull request #4171: URL: https://github.com/apache/hudi/pull/4171#issuecomment-983204766 ## CI report: * 7c8ddb42aa8e8ae597c00635c415368860604ead Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] danny0405 commented on pull request #4171: Revert "[HUDI-2856] Bit cask disk map delete modified"

danny0405 commented on pull request #4171: URL: https://github.com/apache/hudi/pull/4171#issuecomment-983228109 And the line that throws exception ~line 270 should not be affected by #4116 right ? -- This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [hudi] danny0405 commented on pull request #4171: Revert "[HUDI-2856] Bit cask disk map delete modified"

danny0405 commented on pull request #4171: URL: https://github.com/apache/hudi/pull/4171#issuecomment-983227239 Can we give some explanation why this exception throws with the #4116, the code logic does not change ? The only difference is that the new code may throw exception in the `final

[jira] [Commented] (HUDI-2886) Certify metadata table using large-scale cluster testing

[ https://issues.apache.org/jira/browse/HUDI-2886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17451471#comment-17451471 ] Manoj Govindassamy commented on HUDI-2886: -- Applied a workaround of retrying S3 I

[jira] [Commented] (HUDI-2886) Certify metadata table using large-scale cluster testing

[ https://issues.apache.org/jira/browse/HUDI-2886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17451470#comment-17451470 ] Manoj Govindassamy commented on HUDI-2886: -- MOR test run with more than 10 commit

[GitHub] [hudi] hudi-bot removed a comment on pull request #4167: [HUDI-2880] Fixing loading of props from default dir

hudi-bot removed a comment on pull request #4167: URL: https://github.com/apache/hudi/pull/4167#issuecomment-983188356 ## CI report: * bf0309fdfef570903c582da51b7a0a46efb9da2d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4167: [HUDI-2880] Fixing loading of props from default dir

hudi-bot commented on pull request #4167: URL: https://github.com/apache/hudi/pull/4167#issuecomment-983221885 ## CI report: * 1c3bad0199b6744fecc1f3638ea30aac788b35bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] vinothchandar commented on pull request #3173: [HUDI-1951] Add bucket hash index, compatible with the hive bucket

vinothchandar commented on pull request #3173: URL: https://github.com/apache/hudi/pull/3173#issuecomment-983211429 @minihippo As the 0.10 voting goes on, lets work together to land this. I ll make sometime later this or over weekend. Thanks for your patience. -- This is an automated mes

[GitHub] [hudi] hudi-bot removed a comment on pull request #4171: Revert "[HUDI-2856] Bit cask disk map delete modified"

hudi-bot removed a comment on pull request #4171: URL: https://github.com/apache/hudi/pull/4171#issuecomment-983203663 ## CI report: * 7c8ddb42aa8e8ae597c00635c415368860604ead UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4171: Revert "[HUDI-2856] Bit cask disk map delete modified"

hudi-bot commented on pull request #4171: URL: https://github.com/apache/hudi/pull/4171#issuecomment-983204766 ## CI report: * 7c8ddb42aa8e8ae597c00635c415368860604ead Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[hudi] branch master updated: Revert "[HUDI-2855] Change the default value of 'PAYLOAD_CLASS_NAME' to 'DefaultHoodieRecordPayload' (#4115)" (#4169)

This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 24380c2 Revert "[HUDI-2855] Change the default val

[GitHub] [hudi] yihua merged pull request #4169: [HUDI-2898] Reverting "Change the default value of 'PAYLOAD_CLASS_NAME' to 'DefaultHoodieRecordPayload' (#4115)"

yihua merged pull request #4169: URL: https://github.com/apache/hudi/pull/4169 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[GitHub] [hudi] hudi-bot commented on pull request #4171: Revert "[HUDI-2856] Bit cask disk map delete modified"

hudi-bot commented on pull request #4171: URL: https://github.com/apache/hudi/pull/4171#issuecomment-983203663 ## CI report: * 7c8ddb42aa8e8ae597c00635c415368860604ead UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[hudi] 01/01: Revert "[HUDI-2856] Bit cask disk map delete modified (#4116)"

This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch revert-4116-BitCaskDiskMap_delete_modified in repository https://gitbox.apache.org/repos/asf/hudi.git commit 7c8ddb42aa8e8ae597c00635c415368860604ead Author: Y Ethan Guo AuthorDate: Tue Nov 30 17:

[GitHub] [hudi] yihua opened a new pull request #4171: Revert "[HUDI-2856] Bit cask disk map delete modified"

yihua opened a new pull request #4171: URL: https://github.com/apache/hudi/pull/4171 Reverts apache/hudi#4116 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

[hudi] branch revert-4116-BitCaskDiskMap_delete_modified created (now 7c8ddb4)

This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch revert-4116-BitCaskDiskMap_delete_modified in repository https://gitbox.apache.org/repos/asf/hudi.git. at 7c8ddb4 Revert "[HUDI-2856] Bit cask disk map delete modified (#4116)" This branch

[GitHub] [hudi] yihua commented on pull request #4116: [HUDI-2856] Bit cask disk map delete modified

yihua commented on pull request #4116: URL: https://github.com/apache/hudi/pull/4116#issuecomment-983202109 Reverting this change for 0.10.0-rc3 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [hudi] xiarixiaoyao commented on issue #4135: [SUPPORT] Zordering clustering on a moderate size dataset taking large amounts of time.

xiarixiaoyao commented on issue #4135: URL: https://github.com/apache/hudi/issues/4135#issuecomment-983199927 @rubenssoto Thank you for your feedback createOptimizedDataFrameByMapValue at RDDSpatialCurveOptimizationSortPartitioner.java:83 should support running in parallel， let me chec

[jira] [Updated] (HUDI-2369) Blog on bulk insert sort modes

[ https://issues.apache.org/jira/browse/HUDI-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2369: -- Fix Version/s: 0.11.0 (was: 0.10.0) > Blog on bulk insert sort mo

[jira] [Reopened] (HUDI-2369) Blog on bulk insert sort modes

[ https://issues.apache.org/jira/browse/HUDI-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reopened HUDI-2369: --- Assignee: sivabalan narayanan > Blog on bulk insert sort modes > -

[GitHub] [hudi] leesf commented on a change in pull request #4152: [HUDI-2881] Compact the file group with larger log files to reduce wr…

leesf commented on a change in pull request #4152: URL: https://github.com/apache/hudi/pull/4152#discussion_r759787128 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/compact/strategy/LogFileSizeThresholdBasedCompactionStrategy.java ## @

[jira] [Updated] (HUDI-2880) Error reading in properties from dfs in Spark Shell

[ https://issues.apache.org/jira/browse/HUDI-2880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2880: -- Fix Version/s: 0.10.0 (was: 0.11.0) > Error reading in properties

[GitHub] [hudi] hudi-bot removed a comment on pull request #4167: [HUDI-2880] Fixing loading of props from default dir

hudi-bot removed a comment on pull request #4167: URL: https://github.com/apache/hudi/pull/4167#issuecomment-983187111 ## CI report: * bf0309fdfef570903c582da51b7a0a46efb9da2d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4167: [HUDI-2880] Fixing loading of props from default dir

hudi-bot commented on pull request #4167: URL: https://github.com/apache/hudi/pull/4167#issuecomment-983188356 ## CI report: * bf0309fdfef570903c582da51b7a0a46efb9da2d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4167: [HUDI-2880] Fixing loading of props from default dir

hudi-bot removed a comment on pull request #4167: URL: https://github.com/apache/hudi/pull/4167#issuecomment-982858325 ## CI report: * bf0309fdfef570903c582da51b7a0a46efb9da2d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] nsivabalan commented on pull request #4169: [HUDI-2898] Reverting "Change the default value of 'PAYLOAD_CLASS_NAME' to 'DefaultHoodieRecordPayload' (#4115)"

nsivabalan commented on pull request #4169: URL: https://github.com/apache/hudi/pull/4169#issuecomment-983187027 Here is our reasoning. Our existing tests did not catch this bug. And so, feeling skeptical there could be few other issues. Would prefer to revert it. -- This is an automat

[GitHub] [hudi] hudi-bot commented on pull request #4167: [HUDI-2880] Fixing loading of props from default dir

hudi-bot commented on pull request #4167: URL: https://github.com/apache/hudi/pull/4167#issuecomment-983187111 ## CI report: * bf0309fdfef570903c582da51b7a0a46efb9da2d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] YuweiXiao commented on a change in pull request #3173: [HUDI-1951] Add bucket hash index, compatible with the hive bucket

YuweiXiao commented on a change in pull request #3173: URL: https://github.com/apache/hudi/pull/3173#discussion_r759777178 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieIndexConfig.java ## @@ -200,6 +209,48 @@ .defaultValue("tru

[GitHub] [hudi] YuweiXiao commented on a change in pull request #3173: [HUDI-1951] Add bucket hash index, compatible with the hive bucket

YuweiXiao commented on a change in pull request #3173: URL: https://github.com/apache/hudi/pull/3173#discussion_r759777677 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieIndexConfig.java ## @@ -54,7 +63,7 @@ .key("hoodie.index.ty

[GitHub] [hudi] YuweiXiao commented on a change in pull request #3173: [HUDI-1951] Add bucket hash index, compatible with the hive bucket

YuweiXiao commented on a change in pull request #3173: URL: https://github.com/apache/hudi/pull/3173#discussion_r759777178 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieIndexConfig.java ## @@ -200,6 +209,48 @@ .defaultValue("tru

  1   2   3   >