[GitHub] [hudi] hudi-bot commented on pull request #4187: [HUDI-2912] Fix CompactionPlanOperator typo

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4187: URL: https://github.com/apache/hudi/pull/4187#issuecomment-984366349 ## CI report: * 548a418f486494091e1001aec0a733fd89ca8fbe Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4187: [HUDI-2912] Fix CompactionPlanOperator typo

2021-12-01 Thread GitBox
hudi-bot removed a comment on pull request #4187: URL: https://github.com/apache/hudi/pull/4187#issuecomment-984335994 ## CI report: * 548a418f486494091e1001aec0a733fd89ca8fbe Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4189: [HUDI-2913] Disable auto clean in writer task

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4189: URL: https://github.com/apache/hudi/pull/4189#issuecomment-984348912 ## CI report: * fa44ae507c0b013f0ebe69c58d06bc25f2bfe6ec Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4189: [HUDI-2913] Disable auto clean in writer task

2021-12-01 Thread GitBox
hudi-bot removed a comment on pull request #4189: URL: https://github.com/apache/hudi/pull/4189#issuecomment-984347424 ## CI report: * fa44ae507c0b013f0ebe69c58d06bc25f2bfe6ec UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4189: [HUDI-2913] Disable auto clean in writer task

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4189: URL: https://github.com/apache/hudi/pull/4189#issuecomment-984347424 ## CI report: * fa44ae507c0b013f0ebe69c58d06bc25f2bfe6ec UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[GitHub] [hudi] danny0405 commented on a change in pull request #4181: [HUDI-2900] Fix corrupt block end position

2021-12-01 Thread GitBox
danny0405 commented on a change in pull request #4181: URL: https://github.com/apache/hudi/pull/4181#discussion_r760809572 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java ## @@ -284,7 +284,7 @@ private HoodieLogBlock createCorr

[jira] [Updated] (HUDI-2913) Disable auto clean in writer task

2021-12-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2913: - Labels: pull-request-available (was: ) > Disable auto clean in writer task >

[GitHub] [hudi] yuzhaojing opened a new pull request #4189: [HUDI-2913] Disable auto clean in writer task

2021-12-01 Thread GitBox
yuzhaojing opened a new pull request #4189: URL: https://github.com/apache/hudi/pull/4189 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purp

[GitHub] [hudi] h7kanna opened a new issue #4188: [SUPPORT] NullPointerException in HoodieROTablePathFilter Hudi 0.10.0

2021-12-01 Thread GitBox
h7kanna opened a new issue #4188: URL: https://github.com/apache/hudi/issues/4188 **Describe the problem you faced** NullPointerException in HoodieROTablePathFilter while querying Hudi table using 0.10.0 that is working with 0.9.0 **To Reproduce** Steps to reproduce the

[GitHub] [hudi] hudi-bot commented on pull request #4179: Fix HoodieSqlUtils.formatQueryInstant timestamp variable bug

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4179: URL: https://github.com/apache/hudi/pull/4179#issuecomment-984341438 ## CI report: * ffdf5ee6c364d06cf3dd40f523b36c4eadad24eb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4179: Fix HoodieSqlUtils.formatQueryInstant timestamp variable bug

2021-12-01 Thread GitBox
hudi-bot removed a comment on pull request #4179: URL: https://github.com/apache/hudi/pull/4179#issuecomment-984313491 ## CI report: * ffdf5ee6c364d06cf3dd40f523b36c4eadad24eb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[jira] [Created] (HUDI-2913) Disable auto clean in writer task

2021-12-01 Thread yuzhaojing (Jira)
yuzhaojing created HUDI-2913: Summary: Disable auto clean in writer task Key: HUDI-2913 URL: https://issues.apache.org/jira/browse/HUDI-2913 Project: Apache Hudi Issue Type: Improvement

[GitHub] [hudi] hudi-bot commented on pull request #4186: [HUDI-2904] WIP Fix metadata archive issues

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4186: URL: https://github.com/apache/hudi/pull/4186#issuecomment-984337129 ## CI report: * 8a185b85ce1f42b5bdec94ba676abc44ce0defa4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4186: [HUDI-2904] WIP Fix metadata archive issues

2021-12-01 Thread GitBox
hudi-bot removed a comment on pull request #4186: URL: https://github.com/apache/hudi/pull/4186#issuecomment-984311498 ## CI report: * 8a185b85ce1f42b5bdec94ba676abc44ce0defa4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot removed a comment on pull request #4187: [HUDI-2912] Fix CompactionPlanOperator typo

2021-12-01 Thread GitBox
hudi-bot removed a comment on pull request #4187: URL: https://github.com/apache/hudi/pull/4187#issuecomment-984334873 ## CI report: * 548a418f486494091e1001aec0a733fd89ca8fbe UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4187: [HUDI-2912] Fix CompactionPlanOperator typo

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4187: URL: https://github.com/apache/hudi/pull/4187#issuecomment-984335994 ## CI report: * 548a418f486494091e1001aec0a733fd89ca8fbe Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot commented on pull request #4187: [HUDI-2912] Fix CompactionPlanOperator typo

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4187: URL: https://github.com/apache/hudi/pull/4187#issuecomment-984334873 ## CI report: * 548a418f486494091e1001aec0a733fd89ca8fbe UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[jira] [Updated] (HUDI-2912) Fix CompactionPlanOperator typo

2021-12-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2912: - Labels: pull-request-available (was: ) > Fix CompactionPlanOperator typo > --

[GitHub] [hudi] yuzhaojing opened a new pull request #4187: [HUDI-2912] Fix CompactionPlanOperator typo

2021-12-01 Thread GitBox
yuzhaojing opened a new pull request #4187: URL: https://github.com/apache/hudi/pull/4187 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purp

[jira] [Created] (HUDI-2912) Fix CompactionPlanOperator typo

2021-12-01 Thread yuzhaojing (Jira)
yuzhaojing created HUDI-2912: Summary: Fix CompactionPlanOperator typo Key: HUDI-2912 URL: https://issues.apache.org/jira/browse/HUDI-2912 Project: Apache Hudi Issue Type: Improvement C

[GitHub] [hudi] Gatsby-Lee commented on issue #2544: [SUPPORT]failed to read timestamp column in version 0.7.0 even when HIVE_SUPPORT_TIMESTAMP is enabled

2021-12-01 Thread GitBox
Gatsby-Lee commented on issue #2544: URL: https://github.com/apache/hudi/issues/2544#issuecomment-984330238 @codope Can you tell me where I can find the commit for this fix? And, do you know if there is any downside of setting this config? "hoodie.datasource.hive_sync.support_ti

[GitHub] [hudi] Gatsby-Lee commented on issue #2509: [SUPPORT] Hudi Spark DataSource saves TimestampType as bigInt

2021-12-01 Thread GitBox
Gatsby-Lee commented on issue #2509: URL: https://github.com/apache/hudi/issues/2509#issuecomment-984328231 AWS Glue3 + Spark: 3.1.1-amzn-0 + Hive: 2.3.7-amzn-4 + Hudi: 0.9 I had this issue. Although I can see timestamp type, the type I see through AWS Athena was bigint.

[GitHub] [hudi] yanghua commented on pull request #3671: [HUDI-2418] add HiveSchemaProvider

2021-12-01 Thread GitBox
yanghua commented on pull request #3671: URL: https://github.com/apache/hudi/pull/3671#issuecomment-984318178 sorry for the late reply. Will have a final check soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hudi] leesf commented on pull request #4179: Fix HoodieSqlUtils.formatQueryInstant timestamp variable bug

2021-12-01 Thread GitBox
leesf commented on pull request #4179: URL: https://github.com/apache/hudi/pull/4179#issuecomment-984313456 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [hudi] hudi-bot removed a comment on pull request #4179: Fix HoodieSqlUtils.formatQueryInstant timestamp variable bug

2021-12-01 Thread GitBox
hudi-bot removed a comment on pull request #4179: URL: https://github.com/apache/hudi/pull/4179#issuecomment-983721370 ## CI report: * ffdf5ee6c364d06cf3dd40f523b36c4eadad24eb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4179: Fix HoodieSqlUtils.formatQueryInstant timestamp variable bug

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4179: URL: https://github.com/apache/hudi/pull/4179#issuecomment-984313491 ## CI report: * ffdf5ee6c364d06cf3dd40f523b36c4eadad24eb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4186: [HUDI-2904] WIP Fix metadata archive issues

2021-12-01 Thread GitBox
hudi-bot removed a comment on pull request #4186: URL: https://github.com/apache/hudi/pull/4186#issuecomment-984310591 ## CI report: * 8a185b85ce1f42b5bdec94ba676abc44ce0defa4 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4186: [HUDI-2904] WIP Fix metadata archive issues

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4186: URL: https://github.com/apache/hudi/pull/4186#issuecomment-984311498 ## CI report: * 8a185b85ce1f42b5bdec94ba676abc44ce0defa4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot commented on pull request #4186: [HUDI-2904] WIP Fix metadata archive issues

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4186: URL: https://github.com/apache/hudi/pull/4186#issuecomment-984310591 ## CI report: * 8a185b85ce1f42b5bdec94ba676abc44ce0defa4 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[jira] [Updated] (HUDI-2904) Failed to archive commits due to no such file in metadata

2021-12-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2904: - Labels: pull-request-available (was: ) > Failed to archive commits due to no such file in metadat

[GitHub] [hudi] rmahindra123 opened a new pull request #4186: [HUDI-2904] WIP Fix metadata archive issues

2021-12-01 Thread GitBox
rmahindra123 opened a new pull request #4186: URL: https://github.com/apache/hudi/pull/4186 When metadata is enabled, and with Single writer, an async service such as clustering on the data table can cause archival at the same time the regular writer may trigger archival on the metadata ta

[hudi] branch master updated (5284730 -> 772f5ca)

2021-12-01 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 5284730 [HUDI-2881] Compact the file group with larger log files to reduce write amplification (#4152) add 772f5c

[GitHub] [hudi] yihua merged pull request #4183: [HUDI-2908] Fixed partitions produced by layout optimization in case order-by key is composed of a single column

2021-12-01 Thread GitBox
yihua merged pull request #4183: URL: https://github.com/apache/hudi/pull/4183 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[GitHub] [hudi] Gatsby-Lee commented on issue #2544: [SUPPORT]failed to read timestamp column in version 0.7.0 even when HIVE_SUPPORT_TIMESTAMP is enabled

2021-12-01 Thread GitBox
Gatsby-Lee commented on issue #2544: URL: https://github.com/apache/hudi/issues/2544#issuecomment-984291838 Thank you!! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] hudi-bot commented on pull request #4185: [HUDI-2894][HUDI-2905] Metadata table - avoiding key lookup failures on base files over S3

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4185: URL: https://github.com/apache/hudi/pull/4185#issuecomment-984289273 ## CI report: * 5602e0b15b5d3ca9ddd30b3f091439a03d951568 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4185: [HUDI-2894][HUDI-2905] Metadata table - avoiding key lookup failures on base files over S3

2021-12-01 Thread GitBox
hudi-bot removed a comment on pull request #4185: URL: https://github.com/apache/hudi/pull/4185#issuecomment-984269615 ## CI report: * 5602e0b15b5d3ca9ddd30b3f091439a03d951568 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot removed a comment on pull request #4173: [MINOR] Mitigate CI jobs timeout issues

2021-12-01 Thread GitBox
hudi-bot removed a comment on pull request #4173: URL: https://github.com/apache/hudi/pull/4173#issuecomment-984238562 ## CI report: * 66c6b0d67d07d6eed59b3653c91dbacd87c05501 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4173: [MINOR] Mitigate CI jobs timeout issues

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4173: URL: https://github.com/apache/hudi/pull/4173#issuecomment-984281684 ## CI report: * dfad8ecf4258b000562b3b188e774d926aea6a1e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[jira] [Updated] (HUDI-2904) Failed to archive commits due to no such file in metadata

2021-12-01 Thread Rajesh Mahindra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Mahindra updated HUDI-2904: -- Priority: Blocker (was: Major) > Failed to archive commits due to no such file in metadata > --

[jira] [Assigned] (HUDI-2904) Failed to archive commits due to no such file in metadata

2021-12-01 Thread Rajesh Mahindra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Mahindra reassigned HUDI-2904: - Assignee: Rajesh Mahindra > Failed to archive commits due to no such file in metadata > -

[GitHub] [hudi] hudi-bot removed a comment on pull request #4185: [HUDI-2894][HUDI-2905] Metadata table - avoiding key lookup failures on base files over S3

2021-12-01 Thread GitBox
hudi-bot removed a comment on pull request #4185: URL: https://github.com/apache/hudi/pull/4185#issuecomment-984268716 ## CI report: * 5602e0b15b5d3ca9ddd30b3f091439a03d951568 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4185: [HUDI-2894][HUDI-2905] Metadata table - avoiding key lookup failures on base files over S3

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4185: URL: https://github.com/apache/hudi/pull/4185#issuecomment-984269615 ## CI report: * 5602e0b15b5d3ca9ddd30b3f091439a03d951568 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot commented on pull request #4185: [HUDI-2894][HUDI-2905] Metadata table - avoiding key lookup failures on base files over S3

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4185: URL: https://github.com/apache/hudi/pull/4185#issuecomment-984268716 ## CI report: * 5602e0b15b5d3ca9ddd30b3f091439a03d951568 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[jira] [Updated] (HUDI-2894) Metadata table read after compaction fails in S3

2021-12-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2894: - Labels: pull-request-available (was: ) > Metadata table read after compaction fails in S3 > -

[GitHub] [hudi] manojpec opened a new pull request #4185: [HUDI-2894] Metadata table - avoiding key lookup failures on base files over S3

2021-12-01 Thread GitBox
manojpec opened a new pull request #4185: URL: https://github.com/apache/hudi/pull/4185 ## What is the purpose of the pull request - Fetching partition files or all partitions from the metadata table is failing when run over S3. Metadata table uses HFile format for the base fi

[GitHub] [hudi] hudi-bot commented on pull request #4182: [MINOR] use catalog schema if can not find table schema

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4182: URL: https://github.com/apache/hudi/pull/4182#issuecomment-984266216 ## CI report: * aef2c9c5d890b808384cfae906b4ea1f722659a0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4182: [MINOR] use catalog schema if can not find table schema

2021-12-01 Thread GitBox
hudi-bot removed a comment on pull request #4182: URL: https://github.com/apache/hudi/pull/4182#issuecomment-984232128 ## CI report: * aef2c9c5d890b808384cfae906b4ea1f722659a0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4181: [HUDI-2900] Fix corrupt block end position

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4181: URL: https://github.com/apache/hudi/pull/4181#issuecomment-984263502 ## CI report: * 9924dc7a8af334d3d641da49e045e0b105ddb2c9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4181: [HUDI-2900] Fix corrupt block end position

2021-12-01 Thread GitBox
hudi-bot removed a comment on pull request #4181: URL: https://github.com/apache/hudi/pull/4181#issuecomment-984226634 ## CI report: * 9924dc7a8af334d3d641da49e045e0b105ddb2c9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] wangzhongz opened a new issue #4184: [SUPPORT]parquet is not a Parquet file (too small length:4)

2021-12-01 Thread GitBox
wangzhongz opened a new issue #4184: URL: https://github.com/apache/hudi/issues/4184 MOR + Spark **Environment Description** * Hudi version : 0.9 * Spark version : spark2 **Stacktrace** ```Add the stacktrace of the error.``` ![image](https://user-

[GitHub] [hudi] hudi-bot commented on pull request #4183: [HUDI-2908] Fixed partitions produced by layout optimization in case order-by key is composed of a single column

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4183: URL: https://github.com/apache/hudi/pull/4183#issuecomment-984251732 ## CI report: * e28f1f7cc461c327254bbf7c78e4e01985abcf11 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[jira] [Updated] (HUDI-2908) Clustering w/ Layout Optimization enabled, produces incorrect number of partitions

2021-12-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2908: - Labels: pull-request-available (was: ) > Clustering w/ Layout Optimization enabled, produces inco

[GitHub] [hudi] alexeykudinkin opened a new pull request #4183: [HUDI-2908] Fixed partitions produced by layout optimization in case order-by key is composed of a single column

2021-12-01 Thread GitBox
alexeykudinkin opened a new pull request #4183: URL: https://github.com/apache/hudi/pull/4183 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

[jira] [Updated] (HUDI-2911) Writing non-partitioned table produces incorrect "hoodie.properties" file

2021-12-01 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-2911: -- Fix Version/s: 0.11.0 > Writing non-partitioned table produces incorrect "hoodie.properties" fil

[jira] [Updated] (HUDI-2911) Writing non-partitioned table produces incorrect "hoodie.properties" file

2021-12-01 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-2911: -- Priority: Blocker (was: Major) > Writing non-partitioned table produces incorrect "hoodie.prope

[jira] [Created] (HUDI-2911) Writing non-partitioned table produces incorrect "hoodie.properties" file

2021-12-01 Thread Alexey Kudinkin (Jira)
Alexey Kudinkin created HUDI-2911: - Summary: Writing non-partitioned table produces incorrect "hoodie.properties" file Key: HUDI-2911 URL: https://issues.apache.org/jira/browse/HUDI-2911 Project: Apac

[GitHub] [hudi] hudi-bot commented on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-984239555 ## CI report: * c454677b96fab062cf31634426646d741ac9dbe5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-01 Thread GitBox
hudi-bot removed a comment on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-984210665 ## CI report: * c454677b96fab062cf31634426646d741ac9dbe5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4173: [MINOR] Mitigate CI jobs timeout issues

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4173: URL: https://github.com/apache/hudi/pull/4173#issuecomment-984238562 ## CI report: * 66c6b0d67d07d6eed59b3653c91dbacd87c05501 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] gudladona commented on issue #3834: [SUPPORT] - AWS Athena snapshot query fails if there are two or more record array fields in a MoR table

2021-12-01 Thread GitBox
gudladona commented on issue #3834: URL: https://github.com/apache/hudi/issues/3834#issuecomment-984238436 I think this is fixed in https://github.com/apache/parquet-mr/pull/560. Upgrading parquet-avro to >=1.11.0 should address this issue. -- This is an automated message from the Apache

[GitHub] [hudi] hudi-bot removed a comment on pull request #4173: [MINOR] Mitigate CI jobs timeout issues

2021-12-01 Thread GitBox
hudi-bot removed a comment on pull request #4173: URL: https://github.com/apache/hudi/pull/4173#issuecomment-984235437 ## CI report: * 66c6b0d67d07d6eed59b3653c91dbacd87c05501 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] xushiyan commented on pull request #4180: [HUDI-2903] get table schema from the last commit with data written

2021-12-01 Thread GitBox
xushiyan commented on pull request #4180: URL: https://github.com/apache/hudi/pull/4180#issuecomment-984238456 As discussed, let's hold this off. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] hudi-bot commented on pull request #4173: [MINOR] [WIP] Check on TestHBaseIndex

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4173: URL: https://github.com/apache/hudi/pull/4173#issuecomment-984235437 ## CI report: * 66c6b0d67d07d6eed59b3653c91dbacd87c05501 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4173: [MINOR] [WIP] Check on TestHBaseIndex

2021-12-01 Thread GitBox
hudi-bot removed a comment on pull request #4173: URL: https://github.com/apache/hudi/pull/4173#issuecomment-983627070 ## CI report: * 66c6b0d67d07d6eed59b3653c91dbacd87c05501 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4182: [MINOR] use catalog schema if can not find table schema

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4182: URL: https://github.com/apache/hudi/pull/4182#issuecomment-984232128 ## CI report: * aef2c9c5d890b808384cfae906b4ea1f722659a0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4182: [MINOR] use catalog schema if can not find table schema

2021-12-01 Thread GitBox
hudi-bot removed a comment on pull request #4182: URL: https://github.com/apache/hudi/pull/4182#issuecomment-984231059 ## CI report: * aef2c9c5d890b808384cfae906b4ea1f722659a0 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4182: [MINOR] use catalog schema if can not find table schema

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4182: URL: https://github.com/apache/hudi/pull/4182#issuecomment-984231059 ## CI report: * aef2c9c5d890b808384cfae906b4ea1f722659a0 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[GitHub] [hudi] YannByron opened a new pull request #4182: [MINOR] use catalog schema if can not find table schema

2021-12-01 Thread GitBox
YannByron opened a new pull request #4182: URL: https://github.com/apache/hudi/pull/4182 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpo

[jira] [Assigned] (HUDI-2905) Insert crashes in MOR table with NullPointerException from HoodieMergeHandle

2021-12-01 Thread Manoj Govindassamy (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy reassigned HUDI-2905: Assignee: Manoj Govindassamy > Insert crashes in MOR table with NullPointerExceptio

[GitHub] [hudi] hudi-bot removed a comment on pull request #4181: [HUDI-2900] Fix corrupt block end position

2021-12-01 Thread GitBox
hudi-bot removed a comment on pull request #4181: URL: https://github.com/apache/hudi/pull/4181#issuecomment-983907003 ## CI report: * 9924dc7a8af334d3d641da49e045e0b105ddb2c9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4181: [HUDI-2900] Fix corrupt block end position

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4181: URL: https://github.com/apache/hudi/pull/4181#issuecomment-984226634 ## CI report: * 9924dc7a8af334d3d641da49e045e0b105ddb2c9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] danny0405 commented on pull request #4181: [HUDI-2900] Fix corrupt block end position

2021-12-01 Thread GitBox
danny0405 commented on pull request #4181: URL: https://github.com/apache/hudi/pull/4181#issuecomment-984226564 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[hudi] branch master updated (f4c25ba -> 5284730)

2021-12-01 Thread leesf
This is an automated email from the ASF dual-hosted git repository. leesf pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from f4c25ba [HUDI-2880] Fixing loading of props from default dir (#4167) add 5284730 [HUDI-2881] Compact the file gro

[GitHub] [hudi] leesf merged pull request #4152: [HUDI-2881] Compact the file group with larger log files to reduce wr…

2021-12-01 Thread GitBox
leesf merged pull request #4152: URL: https://github.com/apache/hudi/pull/4152 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[GitHub] [hudi] xiarixiaoyao edited a comment on issue #4135: [SUPPORT] Zordering clustering on a moderate size dataset taking large amounts of time.

2021-12-01 Thread GitBox
xiarixiaoyao edited a comment on issue #4135: URL: https://github.com/apache/hudi/issues/4135#issuecomment-983424937 @vinothchandar The current cluster mechanism just can't support concurrency very well. Even if you use ordinary sorting (not z-order / Hilbert), there also exsit this

[GitHub] [hudi] hudi-bot commented on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-984210665 ## CI report: * c454677b96fab062cf31634426646d741ac9dbe5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-01 Thread GitBox
hudi-bot removed a comment on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-983676154 ## CI report: * c454677b96fab062cf31634426646d741ac9dbe5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] xiarixiaoyao commented on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-01 Thread GitBox
xiarixiaoyao commented on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-984209981 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[jira] [Created] (HUDI-2910) Hudi CLI "commits showarchived" throws NPE

2021-12-01 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-2910: --- Summary: Hudi CLI "commits showarchived" throws NPE Key: HUDI-2910 URL: https://issues.apache.org/jira/browse/HUDI-2910 Project: Apache Hudi Issue Type: Bug

[GitHub] [hudi] h7kanna commented on pull request #3944: [HUDI-2495] resolve inconsistent key generation for timestamp types b…

2021-12-01 Thread GitBox
h7kanna commented on pull request #3944: URL: https://github.com/apache/hudi/pull/3944#issuecomment-984179033 @YannByron @leesf I think this broke the keygen. I do not know the context of this change. Can you please verify this https://issues.apache.org/jira/browse/HUDI-2909. T

[GitHub] [hudi] hudi-bot removed a comment on pull request #4175: [HUDI-2883] Refactor hive sync tool / config to use reflection and standardize configs

2021-12-01 Thread GitBox
hudi-bot removed a comment on pull request #4175: URL: https://github.com/apache/hudi/pull/4175#issuecomment-984079663 ## CI report: * edc7087b751940625db4566dd3f4e9e77bf26aa7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4175: [HUDI-2883] Refactor hive sync tool / config to use reflection and standardize configs

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4175: URL: https://github.com/apache/hudi/pull/4175#issuecomment-984122742 ## CI report: * f74bd3089f57aa8e229065139a8307bd2cf70892 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[jira] [Updated] (HUDI-2909) KeyGenerator is broken in 0.10.0

2021-12-01 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-2909: Priority: Blocker (was: Major) > KeyGenerator is broken in 0.10.0 > ---

[jira] [Created] (HUDI-2909) KeyGenerator is broken in 0.10.0

2021-12-01 Thread Harsha Teja Kanna (Jira)
Harsha Teja Kanna created HUDI-2909: --- Summary: KeyGenerator is broken in 0.10.0 Key: HUDI-2909 URL: https://issues.apache.org/jira/browse/HUDI-2909 Project: Apache Hudi Issue Type: Bug

[GitHub] [hudi] hudi-bot removed a comment on pull request #4175: [HUDI-2883] Refactor hive sync tool / config to use reflection and standardize configs

2021-12-01 Thread GitBox
hudi-bot removed a comment on pull request #4175: URL: https://github.com/apache/hudi/pull/4175#issuecomment-984077767 ## CI report: * edc7087b751940625db4566dd3f4e9e77bf26aa7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4175: [HUDI-2883] Refactor hive sync tool / config to use reflection and standardize configs

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4175: URL: https://github.com/apache/hudi/pull/4175#issuecomment-984079663 ## CI report: * edc7087b751940625db4566dd3f4e9e77bf26aa7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4175: [HUDI-2883] Refactor hive sync tool / config to use reflection and standardize configs

2021-12-01 Thread GitBox
hudi-bot removed a comment on pull request #4175: URL: https://github.com/apache/hudi/pull/4175#issuecomment-983457360 ## CI report: * edc7087b751940625db4566dd3f4e9e77bf26aa7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4175: [HUDI-2883] Refactor hive sync tool / config to use reflection and standardize configs

2021-12-01 Thread GitBox
hudi-bot commented on pull request #4175: URL: https://github.com/apache/hudi/pull/4175#issuecomment-984077767 ## CI report: * edc7087b751940625db4566dd3f4e9e77bf26aa7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] nsivabalan commented on pull request #4034: [HUDI-2793] Fixing deltastreamer checkpoint fetch/copy over

2021-12-01 Thread GitBox
nsivabalan commented on pull request #4034: URL: https://github.com/apache/hudi/pull/4034#issuecomment-983970030 yes, its lazy evaluation. once first entry is found, we may not process others. -- This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Updated] (HUDI-2908) Clustering w/ Layout Optimization enabled, produces incorrect number of partitions

2021-12-01 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-2908: -- Priority: Major (was: Blocker) > Clustering w/ Layout Optimization enabled, produces incorrect

[jira] [Updated] (HUDI-2908) Clustering w/ Layout Optimization enabled, produces incorrect number of partitions

2021-12-01 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-2908: -- Description: Currently when clustering w/ Layout Optimization enabled (both Z-order/Hilbert) in

[jira] [Updated] (HUDI-2908) Clustering w/ Layout Optimization enabled, produces incorrect number of partitions

2021-12-01 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-2908: -- Priority: Blocker (was: Major) > Clustering w/ Layout Optimization enabled, produces incorrect

[jira] [Updated] (HUDI-2908) Clustering w/ Layout Optimization enabled, produces incorrect number of partitions

2021-12-01 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-2908: -- Fix Version/s: 0.10.0 > Clustering w/ Layout Optimization enabled, produces incorrect number of

[jira] [Assigned] (HUDI-2908) Clustering w/ Layout Optimization enabled, produces incorrect number of partitions

2021-12-01 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin reassigned HUDI-2908: - Assignee: Alexey Kudinkin > Clustering w/ Layout Optimization enabled, produces incorrect

[jira] [Created] (HUDI-2908) Clustering w/ Layout Optimization enabled, produces incorrect number of partitions

2021-12-01 Thread Alexey Kudinkin (Jira)
Alexey Kudinkin created HUDI-2908: - Summary: Clustering w/ Layout Optimization enabled, produces incorrect number of partitions Key: HUDI-2908 URL: https://issues.apache.org/jira/browse/HUDI-2908 Proj

[jira] [Created] (HUDI-2907) Add a table service to validate states

2021-12-01 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-2907: --- Summary: Add a table service to validate states Key: HUDI-2907 URL: https://issues.apache.org/jira/browse/HUDI-2907 Project: Apache Hudi Issue Type: New Feature

[jira] [Created] (HUDI-2906) Add a repair command to clean up duplicate/uncommitted data files in a table

2021-12-01 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-2906: --- Summary: Add a repair command to clean up duplicate/uncommitted data files in a table Key: HUDI-2906 URL: https://issues.apache.org/jira/browse/HUDI-2906 Project: Apache Hudi

[jira] [Updated] (HUDI-2907) Add a table service to validate data files against timeline

2021-12-01 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-2907: Summary: Add a table service to validate data files against timeline (was: Add a table service to validate

[jira] [Assigned] (HUDI-2906) Add a repair command to clean up duplicate/uncommitted data files in a table

2021-12-01 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reassigned HUDI-2906: --- Assignee: Ethan Guo > Add a repair command to clean up duplicate/uncommitted data files in a table >

[GitHub] [hudi] yihua commented on a change in pull request #4166: [MINOR] Adding verbose output for metadata validate files command

2021-12-01 Thread GitBox
yihua commented on a change in pull request #4166: URL: https://github.com/apache/hudi/pull/4166#discussion_r760447080 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/MetadataCommand.java ## @@ -297,12 +297,20 @@ public String validateFiles( row[0] =

  1   2   3   >