[GitHub] [hudi] huangweifeng7 commented on issue #143: Tracking ticket for folks to be added to slack group

2022-01-06 Thread GitBox
huangweifeng7 commented on issue #143: URL: https://github.com/apache/hudi/issues/143#issuecomment-1006357688 Please add me to slack group Email:huangweifeng_n...@126.com Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] hudi-bot commented on pull request #4471: [HUDI-3125] spark-sql write timestamp directly

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4471: URL: https://github.com/apache/hudi/pull/4471#issuecomment-1006360315 ## CI report: * a5dcf171a39b236a74b9a70b0eb0b49e74ebc3b5 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4471: [HUDI-3125] spark-sql write timestamp directly

2022-01-06 Thread GitBox
hudi-bot removed a comment on pull request #4471: URL: https://github.com/apache/hudi/pull/4471#issuecomment-1006331887 ## CI report: * 29b1742747a4195db690d09f09de972ab7f409db Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] leesf commented on a change in pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-06 Thread GitBox
leesf commented on a change in pull request #4514: URL: https://github.com/apache/hudi/pull/4514#discussion_r779368601 ## File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/HoodieSqlCommonUtils.scala ## @@ -0,0 +1,316 @@ +/* + * License

[GitHub] [hudi] leesf commented on pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-06 Thread GitBox
leesf commented on pull request #4514: URL: https://github.com/apache/hudi/pull/4514#issuecomment-1006364350 > Let me make another pass at all the pom changes. That seems to be main thing here. In the meantime, could you clarify these comments? > > Also have you tested these changes

[GitHub] [hudi] leesf commented on a change in pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-06 Thread GitBox
leesf commented on a change in pull request #4514: URL: https://github.com/apache/hudi/pull/4514#discussion_r779369728 ## File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/HoodieSqlCommonUtils.scala ## @@ -0,0 +1,316 @@ +/* + * License

[GitHub] [hudi] hudi-bot commented on pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4514: URL: https://github.com/apache/hudi/pull/4514#issuecomment-1006367031 ## CI report: * ddc3af0c32bafef6b10c32c43132df32a5f7d83c UNKNOWN * e1ba726105dfa7ae07d802546c71a0cf1ad8b172 UNKNOWN * 306e7d462959e0249e230f60c2e9ea6602342e08 UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-06 Thread GitBox
hudi-bot removed a comment on pull request #4514: URL: https://github.com/apache/hudi/pull/4514#issuecomment-1006323042 ## CI report: * ddc3af0c32bafef6b10c32c43132df32a5f7d83c UNKNOWN * e1ba726105dfa7ae07d802546c71a0cf1ad8b172 UNKNOWN * 306e7d462959e0249e230f60c2e9ea6602342

[GitHub] [hudi] hudi-bot removed a comment on pull request #4521: [HUDI-3183] Wrong result of HoodieArchivedTimeline loadInstants with TimeRangeFilter

2022-01-06 Thread GitBox
hudi-bot removed a comment on pull request #4521: URL: https://github.com/apache/hudi/pull/4521#issuecomment-1006329173 ## CI report: * 16d5dc61ae5c7a9962fc3756720d8262bdadf6b9 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4521: [HUDI-3183] Wrong result of HoodieArchivedTimeline loadInstants with TimeRangeFilter

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4521: URL: https://github.com/apache/hudi/pull/4521#issuecomment-1006367102 ## CI report: * d708467de740637a394375335181979a343979bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] codope commented on a change in pull request #2768: [HUDI-485]: corrected the check for incremental sql

2022-01-06 Thread GitBox
codope commented on a change in pull request #2768: URL: https://github.com/apache/hudi/pull/2768#discussion_r779376837 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/TestHiveIncrementalPuller.java ## @@ -41,4 +67,84 @@ public void testInitHiveIncremental

[GitHub] [hudi] leesf commented on a change in pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-06 Thread GitBox
leesf commented on a change in pull request #4514: URL: https://github.com/apache/hudi/pull/4514#discussion_r779377180 ## File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/HoodieSqlCommonUtils.scala ## @@ -0,0 +1,316 @@ +/* + * License

[GitHub] [hudi] xushiyan commented on pull request #4515: [HUDI-3158] Reduce warn logs in Spark SQL INSERT OVERWRITE

2022-01-06 Thread GitBox
xushiyan commented on pull request #4515: URL: https://github.com/apache/hudi/pull/4515#issuecomment-1006374242 @dongkelun the warn log comes from clustering planning. can you help clarify how would this change avoid the repeated warn logs? -- This is an automated message from the Apache

[GitHub] [hudi] dongkelun commented on pull request #4515: [HUDI-3158] Reduce warn logs in Spark SQL INSERT OVERWRITE

2022-01-06 Thread GitBox
dongkelun commented on pull request #4515: URL: https://github.com/apache/hudi/pull/4515#issuecomment-1006384615 > @dongkelun the warn log comes from clustering planning. can you help clarify how would this change avoid the repeated warn logs? Hello, the reason for the warning is tha

[GitHub] [hudi] hudi-bot commented on pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4514: URL: https://github.com/apache/hudi/pull/4514#issuecomment-1006386632 ## CI report: * ddc3af0c32bafef6b10c32c43132df32a5f7d83c UNKNOWN * e1ba726105dfa7ae07d802546c71a0cf1ad8b172 UNKNOWN * 306e7d462959e0249e230f60c2e9ea6602342e08 UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-06 Thread GitBox
hudi-bot removed a comment on pull request #4514: URL: https://github.com/apache/hudi/pull/4514#issuecomment-1006367031 ## CI report: * ddc3af0c32bafef6b10c32c43132df32a5f7d83c UNKNOWN * e1ba726105dfa7ae07d802546c71a0cf1ad8b172 UNKNOWN * 306e7d462959e0249e230f60c2e9ea6602342

[GitHub] [hudi] hudi-bot removed a comment on pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

2022-01-06 Thread GitBox
hudi-bot removed a comment on pull request #4352: URL: https://github.com/apache/hudi/pull/4352#issuecomment-1006153120 ## CI report: * 235981abd20a498a3e29e98ce0eda9de35018f99 UNKNOWN * 486c6886c5b0bd748e3db1c90c886a1b7f6d52e8 Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot commented on pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4352: URL: https://github.com/apache/hudi/pull/4352#issuecomment-1006400478 ## CI report: * 235981abd20a498a3e29e98ce0eda9de35018f99 UNKNOWN * 486c6886c5b0bd748e3db1c90c886a1b7f6d52e8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4516: [WIP][HUDI-1295] Enabling metadata table based index by default for tests

2022-01-06 Thread GitBox
hudi-bot removed a comment on pull request #4516: URL: https://github.com/apache/hudi/pull/4516#issuecomment-1006131415 ## CI report: * 97502fa31dda3b94645631303e134bf0d652c17e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4516: [WIP][HUDI-1295] Enabling metadata table based index by default for tests

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4516: URL: https://github.com/apache/hudi/pull/4516#issuecomment-1006400740 ## CI report: * 97502fa31dda3b94645631303e134bf0d652c17e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4516: [WIP][HUDI-1295] Enabling metadata table based index by default for tests

2022-01-06 Thread GitBox
hudi-bot removed a comment on pull request #4516: URL: https://github.com/apache/hudi/pull/4516#issuecomment-1006400740 ## CI report: * 97502fa31dda3b94645631303e134bf0d652c17e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4516: [WIP][HUDI-1295] Enabling metadata table based index by default for tests

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4516: URL: https://github.com/apache/hudi/pull/4516#issuecomment-1006402795 ## CI report: * 97502fa31dda3b94645631303e134bf0d652c17e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4516: [WIP][HUDI-1295] Enabling metadata table based index by default for tests

2022-01-06 Thread GitBox
hudi-bot removed a comment on pull request #4516: URL: https://github.com/apache/hudi/pull/4516#issuecomment-1006402795 ## CI report: * 97502fa31dda3b94645631303e134bf0d652c17e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4516: [WIP][HUDI-1295] Enabling metadata table based index by default for tests

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4516: URL: https://github.com/apache/hudi/pull/4516#issuecomment-1006404803 ## CI report: * 97502fa31dda3b94645631303e134bf0d652c17e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4514: URL: https://github.com/apache/hudi/pull/4514#issuecomment-1006428061 ## CI report: * ddc3af0c32bafef6b10c32c43132df32a5f7d83c UNKNOWN * e1ba726105dfa7ae07d802546c71a0cf1ad8b172 UNKNOWN * 306e7d462959e0249e230f60c2e9ea6602342e08 UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4514: [HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation

2022-01-06 Thread GitBox
hudi-bot removed a comment on pull request #4514: URL: https://github.com/apache/hudi/pull/4514#issuecomment-1006386632 ## CI report: * ddc3af0c32bafef6b10c32c43132df32a5f7d83c UNKNOWN * e1ba726105dfa7ae07d802546c71a0cf1ad8b172 UNKNOWN * 306e7d462959e0249e230f60c2e9ea6602342

[GitHub] [hudi] hudi-bot commented on pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4352: URL: https://github.com/apache/hudi/pull/4352#issuecomment-1006435512 ## CI report: * 235981abd20a498a3e29e98ce0eda9de35018f99 UNKNOWN * 486c6886c5b0bd748e3db1c90c886a1b7f6d52e8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

2022-01-06 Thread GitBox
hudi-bot removed a comment on pull request #4352: URL: https://github.com/apache/hudi/pull/4352#issuecomment-1006400478 ## CI report: * 235981abd20a498a3e29e98ce0eda9de35018f99 UNKNOWN * 486c6886c5b0bd748e3db1c90c886a1b7f6d52e8 Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] AirToSupply opened a new issue #4522: [SUPPORT] hudi-flink support timestamp-micros

2022-01-06 Thread GitBox
AirToSupply opened a new issue #4522: URL: https://github.com/apache/hudi/issues/4522 **To Reproduce** Steps to reproduce the behavior: 1. The spark engine is used to write data into the hoodie table(PS: There are timestamp type columns in the dataset field). 2. Use the Fli

[GitHub] [hudi] zhangyue19921010 commented on pull request #4521: [HUDI-3183] Wrong result of HoodieArchivedTimeline loadInstants with TimeRangeFilter

2022-01-06 Thread GitBox
zhangyue19921010 commented on pull request #4521: URL: https://github.com/apache/hudi/pull/4521#issuecomment-1006441474 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[jira] [Created] (HUDI-3184) hudi-flink support timestamp-micros

2022-01-06 Thread Well Tang (Jira)
Well Tang created HUDI-3184: --- Summary: hudi-flink support timestamp-micros Key: HUDI-3184 URL: https://issues.apache.org/jira/browse/HUDI-3184 Project: Apache Hudi Issue Type: Improvement

[jira] [Updated] (HUDI-3184) hudi-flink support timestamp-micros

2022-01-06 Thread Well Tang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Well Tang updated HUDI-3184: Fix Version/s: 0.11.0 > hudi-flink support timestamp-micros > --- > >

[GitHub] [hudi] hudi-bot commented on pull request #4521: [HUDI-3183] Wrong result of HoodieArchivedTimeline loadInstants with TimeRangeFilter

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4521: URL: https://github.com/apache/hudi/pull/4521#issuecomment-1006443490 ## CI report: * d708467de740637a394375335181979a343979bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4521: [HUDI-3183] Wrong result of HoodieArchivedTimeline loadInstants with TimeRangeFilter

2022-01-06 Thread GitBox
hudi-bot removed a comment on pull request #4521: URL: https://github.com/apache/hudi/pull/4521#issuecomment-1006367102 ## CI report: * d708467de740637a394375335181979a343979bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[jira] [Assigned] (HUDI-2779) Cache BaseDir if HudiTableNotFound Exception thrown

2022-01-06 Thread Hui An (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui An reassigned HUDI-2779: Assignee: Hui An > Cache BaseDir if HudiTableNotFound Exception thrown > --

[jira] [Closed] (HUDI-3162) Shade AWS dependencies for bundled packages

2022-01-06 Thread Hui An (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui An closed HUDI-3162. Resolution: Duplicate > Shade AWS dependencies for bundled packages > --- >

[jira] [Assigned] (HUDI-3184) hudi-flink support timestamp-micros

2022-01-06 Thread Well Tang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Well Tang reassigned HUDI-3184: --- Attachment: 3.png 2.png 1.png As

[jira] [Updated] (HUDI-3184) hudi-flink support timestamp-micros

2022-01-06 Thread Well Tang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Well Tang updated HUDI-3184: Description: {*}Problem overview{*}: Steps to reproduce the behavior: ①The spark engine is used to

[jira] [Updated] (HUDI-3184) hudi-flink support timestamp-micros

2022-01-06 Thread Well Tang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Well Tang updated HUDI-3184: Status: In Progress (was: Open) > hudi-flink support timestamp-micros > ---

[GitHub] [hudi] zhangyue19921010 commented on pull request #1274: [HUDI-571] Add 'commits show archived' command to CLI

2022-01-06 Thread GitBox
zhangyue19921010 commented on pull request #1274: URL: https://github.com/apache/hudi/pull/1274#issuecomment-1006465368 Hi guys, it seems that there 's a little problem with the regex pattern ` private static final Pattern ARCHIVE_FILE_PATTERN = Pattern.compile("^\\.commits_\\

[GitHub] [hudi] zhangyue19921010 closed pull request #4521: [HUDI-3183] Wrong result of HoodieArchivedTimeline loadInstants with TimeRangeFilter

2022-01-06 Thread GitBox
zhangyue19921010 closed pull request #4521: URL: https://github.com/apache/hudi/pull/4521 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-

[GitHub] [hudi] AirToSupply commented on issue #4522: [SUPPORT] hudi-flink support timestamp-micros

2022-01-06 Thread GitBox
AirToSupply commented on issue #4522: URL: https://github.com/apache/hudi/issues/4522#issuecomment-1006466436 @AirToSupply Thanks, https://issues.apache.org/jira/browse/HUDI-3184 issue created here ~ -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hudi] hudi-bot commented on pull request #4516: [WIP][HUDI-1295] Enabling metadata table based index by default for tests

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4516: URL: https://github.com/apache/hudi/pull/4516#issuecomment-1006470119 ## CI report: * 7e2ec46af829fabeb506d639c54057d32f3c89fa UNKNOWN * 280360f772b47ffab15655e4679b021e151783d7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4516: [WIP][HUDI-1295] Enabling metadata table based index by default for tests

2022-01-06 Thread GitBox
hudi-bot removed a comment on pull request #4516: URL: https://github.com/apache/hudi/pull/4516#issuecomment-1006404803 ## CI report: * 97502fa31dda3b94645631303e134bf0d652c17e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4352: URL: https://github.com/apache/hudi/pull/4352#issuecomment-1006484026 ## CI report: * 235981abd20a498a3e29e98ce0eda9de35018f99 UNKNOWN * ce1b2b4eefdd2e0d46154b2c97dc93abf6982aa0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

2022-01-06 Thread GitBox
hudi-bot removed a comment on pull request #4352: URL: https://github.com/apache/hudi/pull/4352#issuecomment-1006435512 ## CI report: * 235981abd20a498a3e29e98ce0eda9de35018f99 UNKNOWN * 486c6886c5b0bd748e3db1c90c886a1b7f6d52e8 Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot removed a comment on pull request #4521: [HUDI-3183] Wrong result of HoodieArchivedTimeline loadInstants with TimeRangeFilter

2022-01-06 Thread GitBox
hudi-bot removed a comment on pull request #4521: URL: https://github.com/apache/hudi/pull/4521#issuecomment-1006443490 ## CI report: * d708467de740637a394375335181979a343979bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4521: [HUDI-3183] Wrong result of HoodieArchivedTimeline loadInstants with TimeRangeFilter

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4521: URL: https://github.com/apache/hudi/pull/4521#issuecomment-1006493903 ## CI report: * d708467de740637a394375335181979a343979bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] liujinhui1994 closed issue #4027: [SUPPORT] Structured streaming Async clustering IndexOutOfBoundsException

2022-01-06 Thread GitBox
liujinhui1994 closed issue #4027: URL: https://github.com/apache/hudi/issues/4027 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr

[GitHub] [hudi] codope opened a new pull request #4523: [WIP][HUDI-3173] Add INDEX action type and corresponding commit metadata

2022-01-06 Thread GitBox
codope opened a new pull request #4523: URL: https://github.com/apache/hudi/pull/4523 ## What is the purpose of the pull request - Add top level INIDEX action type. - Add supporting methods in HoodieTimeline. - Add index commit metadata which contains index plan. ## Brief

[jira] [Updated] (HUDI-3173) Introduce new INDEX action type

2022-01-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3173: - Labels: pull-request-available (was: ) > Introduce new INDEX action type > --

[GitHub] [hudi] hudi-bot commented on pull request #4523: [WIP][HUDI-3173] Add INDEX action type and corresponding commit metadata

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4523: URL: https://github.com/apache/hudi/pull/4523#issuecomment-1006542672 ## CI report: * 700a87f4f67a1cac8f5b870882ab7b61628b4020 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot removed a comment on pull request #4523: [WIP][HUDI-3173] Add INDEX action type and corresponding commit metadata

2022-01-06 Thread GitBox
hudi-bot removed a comment on pull request #4523: URL: https://github.com/apache/hudi/pull/4523#issuecomment-1006542672 ## CI report: * 700a87f4f67a1cac8f5b870882ab7b61628b4020 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4523: [WIP][HUDI-3173] Add INDEX action type and corresponding commit metadata

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4523: URL: https://github.com/apache/hudi/pull/4523#issuecomment-1006545249 ## CI report: * 700a87f4f67a1cac8f5b870882ab7b61628b4020 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] codope commented on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-06 Thread GitBox
codope commented on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1006547783 > @nsivabalan @codope I have a discussion related to this implement. In this pr, most of work is just to pass `isConsistentLogicalTimestampEnabled` to the method `HoodieAvroUtils.c

[GitHub] [hudi] nochimow commented on issue #4299: [SUPPORT] Upsert performance decreased after 3 years of data loading

2022-01-06 Thread GitBox
nochimow commented on issue #4299: URL: https://github.com/apache/hudi/issues/4299#issuecomment-1006558081 Hi, Still waiting for some updates on this case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [hudi] nsivabalan commented on pull request #4440: [HUDI-3100] Add config for hive conditional sync

2022-01-06 Thread GitBox
nsivabalan commented on pull request #4440: URL: https://github.com/apache/hudi/pull/4440#issuecomment-1006567579 I am ok adding it. I am seeing this as, filling in a gap we had previously. I understand, it is debatable whether to consider as bug fix or not. but I feel, we can add it. -

[jira] [Updated] (HUDI-1850) Read on table fails if the first write to table failed

2022-01-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-1850: -- Sprint: Hudi-Sprint-Jan-3 (was: Hudi 0.10.1 - 2021/01/03) > Read on table fails if the

[GitHub] [hudi] hudi-bot removed a comment on pull request #4523: [WIP][HUDI-3173] Add INDEX action type and corresponding commit metadata

2022-01-06 Thread GitBox
hudi-bot removed a comment on pull request #4523: URL: https://github.com/apache/hudi/pull/4523#issuecomment-1006545249 ## CI report: * 700a87f4f67a1cac8f5b870882ab7b61628b4020 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4523: [WIP][HUDI-3173] Add INDEX action type and corresponding commit metadata

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4523: URL: https://github.com/apache/hudi/pull/4523#issuecomment-1006610017 ## CI report: * 700a87f4f67a1cac8f5b870882ab7b61628b4020 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] garystafford opened a new issue #4524: [SUPPORT] Kafka Connect Sink for Hudi README has Incorrect Command

2022-01-06 Thread GitBox
garystafford opened a new issue #4524: URL: https://github.com/apache/hudi/issues/4524 **Describe the problem you faced** In the current instructions for the [Kafka Connect Sink for Hudi](https://github.com/apache/hudi/blob/master/hudi-kafka-connect/README.md), the command, `conflue

[GitHub] [hudi] hudi-bot removed a comment on pull request #4507: [HUDI-52] Enabling savepoint and restore for MOR table

2022-01-06 Thread GitBox
hudi-bot removed a comment on pull request #4507: URL: https://github.com/apache/hudi/pull/4507#issuecomment-1005353560 ## CI report: * 2968b1793b9b3e339f3a5267984269e02bdf6c83 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4507: [HUDI-52] Enabling savepoint and restore for MOR table

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4507: URL: https://github.com/apache/hudi/pull/4507#issuecomment-1006618506 ## CI report: * 2968b1793b9b3e339f3a5267984269e02bdf6c83 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4507: [HUDI-52] Enabling savepoint and restore for MOR table

2022-01-06 Thread GitBox
hudi-bot removed a comment on pull request #4507: URL: https://github.com/apache/hudi/pull/4507#issuecomment-1006618506 ## CI report: * 2968b1793b9b3e339f3a5267984269e02bdf6c83 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4507: [HUDI-52] Enabling savepoint and restore for MOR table

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4507: URL: https://github.com/apache/hudi/pull/4507#issuecomment-1006620786 ## CI report: * 2968b1793b9b3e339f3a5267984269e02bdf6c83 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] codope commented on a change in pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-06 Thread GitBox
codope commented on a change in pull request #4203: URL: https://github.com/apache/hudi/pull/4203#discussion_r779577663 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/keygen/TimestampBasedAvroKeyGenerator.java ## @@ -125,7 +126,7 @@ public Timestamp

[GitHub] [hudi] nsivabalan commented on pull request #4440: [HUDI-3100] Add config for hive conditional sync

2022-01-06 Thread GitBox
nsivabalan commented on pull request #4440: URL: https://github.com/apache/hudi/pull/4440#issuecomment-1006625969 Ocne you rebase and CI succeeds, I can land this in -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hudi] nsivabalan commented on pull request #4428: [HUDI-44] Adding support to preserve commit metadata for compaction

2022-01-06 Thread GitBox
nsivabalan commented on pull request #4428: URL: https://github.com/apache/hudi/pull/4428#issuecomment-1006642838 Probably we can skip adding it to plan. here is the use-case. lets say a compaction was triggered w/ preserve commit metadata enabled and mid way users thinks that he does n

[GitHub] [hudi] codope merged pull request #4428: [HUDI-44] Adding support to preserve commit metadata for compaction

2022-01-06 Thread GitBox
codope merged pull request #4428: URL: https://github.com/apache/hudi/pull/4428 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr..

[hudi] branch master updated (50fa5a6 -> b6891d2)

2022-01-06 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 50fa5a6 Update HiveIncrementalPuller to configure filesystem (#4431) add b6891d2 [HUDI-44] Adding support to pre

[GitHub] [hudi] hudi-bot commented on pull request #4507: [HUDI-52] Enabling savepoint and restore for MOR table

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4507: URL: https://github.com/apache/hudi/pull/4507#issuecomment-1006659154 ## CI report: * cb52a5afb8fdccd9aadcb50b541a207b1f543886 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4507: [HUDI-52] Enabling savepoint and restore for MOR table

2022-01-06 Thread GitBox
hudi-bot removed a comment on pull request #4507: URL: https://github.com/apache/hudi/pull/4507#issuecomment-1006620786 ## CI report: * 2968b1793b9b3e339f3a5267984269e02bdf6c83 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] parisni opened a new issue #4525: [SUPPORT] Spark metastore schema evolution broken

2022-01-06 Thread GitBox
parisni opened a new issue #4525: URL: https://github.com/apache/hudi/issues/4525 From my experiments, when a given hudi table gets added columns, then all works except spark read from metastore: - hive read metastore -> New Column added - spark read from hudi path -> New column a

[jira] [Updated] (HUDI-3185) HoodieConfig getBoolean method returns null instead of default value

2022-01-06 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3185: -- Priority: Blocker (was: Major) > HoodieConfig getBoolean method returns null instead of default value >

[jira] [Created] (HUDI-3185) HoodieConfig getBoolean method returns null instead of default value

2022-01-06 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-3185: - Summary: HoodieConfig getBoolean method returns null instead of default value Key: HUDI-3185 URL: https://issues.apache.org/jira/browse/HUDI-3185 Project: Apache Hudi

[jira] [Updated] (HUDI-2429) [UMBRELLA] Comprehensive Schema evolution in Hudi

2022-01-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-2429: - Fix Version/s: 0.12.0 (was: 0.11.0) > [UMBRELLA] Comprehensive Schema evolu

[jira] [Updated] (HUDI-2429) [UMBRELLA] Comprehensive Schema evolution in Hudi

2022-01-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-2429: - Fix Version/s: 0.11.0 (was: 0.12.0) > [UMBRELLA] Comprehensive Schema evolu

[jira] [Updated] (HUDI-1896) [UMBRELLA] Implement DeltaStreamer Source for cloud object stores

2022-01-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1896: - Fix Version/s: 1.0.0 > [UMBRELLA] Implement DeltaStreamer Source for cloud object stores > ---

[GitHub] [hudi] codope commented on a change in pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-06 Thread GitBox
codope commented on a change in pull request #4203: URL: https://github.com/apache/hudi/pull/4203#discussion_r779645642 ## File path: hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/keygen/TestTimestampBasedKeyGenerator.java ## @@ -238,6 +238,40 @@ public void test

[jira] [Updated] (HUDI-1046) Support updates during clustering in CoW mode

2022-01-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1046: - Priority: Blocker (was: Major) > Support updates during clustering in CoW mode >

[jira] [Updated] (HUDI-1046) Support updates during clustering in CoW mode

2022-01-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1046: - Fix Version/s: 0.12.0 > Support updates during clustering in CoW mode > --

[jira] [Updated] (HUDI-1045) Support updates during clustering in MoR mode

2022-01-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1045: - Priority: Blocker (was: Major) > Support updates during clustering in MoR mode >

[jira] [Updated] (HUDI-1045) Support updates during clustering in MoR mode

2022-01-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1045: - Fix Version/s: 0.12.0 > Support updates during clustering in MoR mode > --

[GitHub] [hudi] codope merged pull request #4507: [HUDI-52] Enabling savepoint and restore for MOR table

2022-01-06 Thread GitBox
codope merged pull request #4507: URL: https://github.com/apache/hudi/pull/4507 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr..

[hudi] branch master updated: [HUDI-52] Enabling savepoint and restore for MOR table (#4507)

2022-01-06 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 2954027 [HUDI-52] Enabling savepoint and restore

[jira] [Updated] (HUDI-1456) [UMBRELLA] Concurrency Control for Hudi writers and table services

2022-01-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1456: - Summary: [UMBRELLA] Concurrency Control for Hudi writers and table services (was: [UMBRELLA] Conc

[GitHub] [hudi] nsivabalan commented on a change in pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-06 Thread GitBox
nsivabalan commented on a change in pull request #4203: URL: https://github.com/apache/hudi/pull/4203#discussion_r779654146 ## File path: hudi-common/src/main/java/org/apache/hudi/keygen/constant/KeyGeneratorOptions.java ## @@ -56,6 +56,13 @@ .withDocumentation("Partiti

[GitHub] [hudi] hudi-bot commented on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1006705466 ## CI report: * 5b68cadeeec7a6482f9c5a9eeadad1ad816aa962 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-06 Thread GitBox
hudi-bot removed a comment on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1003406987 ## CI report: * 5b68cadeeec7a6482f9c5a9eeadad1ad816aa962 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1006707897 ## CI report: * 5b68cadeeec7a6482f9c5a9eeadad1ad816aa962 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-06 Thread GitBox
hudi-bot removed a comment on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1006705466 ## CI report: * 5b68cadeeec7a6482f9c5a9eeadad1ad816aa962 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[jira] [Updated] (HUDI-3184) hudi-flink support timestamp-micros

2022-01-06 Thread Well Tang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Well Tang updated HUDI-3184: Remaining Estimate: 120h (was: 5h) Original Estimate: 120h (was: 5h) > hudi-flink support timestamp-m

[jira] [Updated] (HUDI-2370) Supports data encryption

2022-01-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-2370: - Fix Version/s: 0.11.0 > Supports data encryption > > > Ke

[jira] [Updated] (HUDI-3173) Introduce new INDEX action type

2022-01-06 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3173: -- Status: In Progress (was: Open) > Introduce new INDEX action type > --- > >

[GitHub] [hudi] nsivabalan commented on a change in pull request #4497: [HUDI-3147] Create pushgateway client based on port

2022-01-06 Thread GitBox
nsivabalan commented on a change in pull request #4497: URL: https://github.com/apache/hudi/pull/4497#discussion_r779678339 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metrics/prometheus/PushGatewayReporter.java ## @@ -51,17 +53,30 @@ protected P

[jira] [Commented] (HUDI-1628) [Umbrella] Improve data locality during ingestion

2022-01-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17470041#comment-17470041 ] Vinoth Chandar commented on HUDI-1628: -- [~guoyihua] assigning to you to drive this fo

[jira] [Assigned] (HUDI-1628) [Umbrella] Improve data locality during ingestion

2022-01-06 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-1628: Assignee: Ethan Guo (was: Thirumalai Raj R) > [Umbrella] Improve data locality during inge

[GitHub] [hudi] hudi-bot commented on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1006747034 ## CI report: * 2c0565c35723d6f5fee071d14299361b321f202e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-06 Thread GitBox
hudi-bot removed a comment on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1006707897 ## CI report: * 5b68cadeeec7a6482f9c5a9eeadad1ad816aa962 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4203: [HUDI-2909] Handle logical type in TimestampBasedKeyGenerator

2022-01-06 Thread GitBox
hudi-bot commented on pull request #4203: URL: https://github.com/apache/hudi/pull/4203#issuecomment-1006751886 ## CI report: * 2c0565c35723d6f5fee071d14299361b321f202e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

  1   2   3   4   >