[jira] [Commented] (HUDI-3793) Spark SQL query fails using hudi-spark-bundle

2022-04-09 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17519914#comment-17519914 ] Raymond Xu commented on HUDI-3793: -- Not seeing the bundling issue from master version. Te

[jira] [Closed] (HUDI-3793) Spark SQL query fails using hudi-spark-bundle

2022-04-09 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-3793. Fix Version/s: (was: 0.11.0) Resolution: Not A Problem > Spark SQL query fails using hudi-spark-b

[GitHub] [hudi] hudi-bot commented on pull request #5270: [HUDI-3835] Add UT for delete in java client

2022-04-09 Thread GitBox
hudi-bot commented on PR #5270: URL: https://github.com/apache/hudi/pull/5270#issuecomment-1093775336 ## CI report: * 89346658ebebb9750cb1c7e3648827519662221c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7940

[jira] [Updated] (HUDI-2866) Get Metadata table bootstrapping in Flink in parity with spark

2022-04-09 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2866: - Fix Version/s: 0.12.0 (was: 0.11.0) > Get Metadata table bootstrapping in Flink in

[GitHub] [hudi] hudi-bot commented on pull request #5268: [HUDI-3807] Add a new config to control the use of metadata index in HoodieBloomIndex

2022-04-09 Thread GitBox
hudi-bot commented on PR #5268: URL: https://github.com/apache/hudi/pull/5268#issuecomment-1093808293 ## CI report: * 230df2998409be2668e9b0637c591c58e9163d6d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7942

[GitHub] [hudi] hudi-bot commented on pull request #5271: [MINOR] Fix typos in the comments of HoodieMergeHandle

2022-04-09 Thread GitBox
hudi-bot commented on PR #5271: URL: https://github.com/apache/hudi/pull/5271#issuecomment-1093823498 ## CI report: * e0505e07e7734c88a554279e4b52e7d3b5a3fa0d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7943

[GitHub] [hudi] hudi-bot commented on pull request #5269: [HUDI-3636] Create new write clients for async table services in DeltaStreamer

2022-04-09 Thread GitBox
hudi-bot commented on PR #5269: URL: https://github.com/apache/hudi/pull/5269#issuecomment-1093839735 ## CI report: * 784c8cf617b0f4eca028205f87067e08bb55457b UNKNOWN * 6f8d22ccc5efbd87ff993a46ea1977355842602f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] arunb2w commented on issue #1483: [SUPPORT] Docker Demo: Failed to Connect to namenode

2022-04-09 Thread GitBox
arunb2w commented on issue #1483: URL: https://github.com/apache/hudi/issues/1483#issuecomment-1093866671 @malanb5 @lamberken I am also facing the same issue and below is the output of docker ps. ``` ╰─ docker ps CONTAINER ID IMAGE

[GitHub] [hudi] nsivabalan commented on pull request #5266: [HUDI-3834] Fixing performance hits in reading Column Stats Index

2022-04-09 Thread GitBox
nsivabalan commented on PR #5266: URL: https://github.com/apache/hudi/pull/5266#issuecomment-1093990776 @alexeykudinkin : there are some CI failures. can you please check it out -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] XuQianJin-Stars opened a new pull request, #5272: [WIP][HUDI-3826] Commands deleting partitions do so incorrectly

2022-04-09 Thread GitBox
XuQianJin-Stars opened a new pull request, #5272: URL: https://github.com/apache/hudi/pull/5272 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

[jira] [Updated] (HUDI-3826) Commands deleting partitions do so incorrectly

2022-04-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3826: - Labels: pull-request-available (was: ) > Commands deleting partitions do so incorrectly > ---

[GitHub] [hudi] hudi-bot commented on pull request #5272: [WIP][HUDI-3826] Commands deleting partitions do so incorrectly

2022-04-09 Thread GitBox
hudi-bot commented on PR #5272: URL: https://github.com/apache/hudi/pull/5272#issuecomment-1094028608 ## CI report: * 76a2f42dbb28b75940f3b8ff28a41d96dae8e4d0 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5272: [WIP][HUDI-3826] Commands deleting partitions do so incorrectly

2022-04-09 Thread GitBox
hudi-bot commented on PR #5272: URL: https://github.com/apache/hudi/pull/5272#issuecomment-1094030798 ## CI report: * 76a2f42dbb28b75940f3b8ff28a41d96dae8e4d0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7945

[GitHub] [hudi] hudi-bot commented on pull request #5272: [WIP][HUDI-3826] Commands deleting partitions do so incorrectly

2022-04-09 Thread GitBox
hudi-bot commented on PR #5272: URL: https://github.com/apache/hudi/pull/5272#issuecomment-1094060718 ## CI report: * 76a2f42dbb28b75940f3b8ff28a41d96dae8e4d0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7945

[jira] [Created] (HUDI-3837) Rat License check is not effective

2022-04-09 Thread Raymond Xu (Jira)
Raymond Xu created HUDI-3837: Summary: Rat License check is not effective Key: HUDI-3837 URL: https://issues.apache.org/jira/browse/HUDI-3837 Project: Apache Hudi Issue Type: Bug Compon

[jira] [Updated] (HUDI-3837) Rat License check is not effective

2022-04-09 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3837: - Fix Version/s: 0.11.0 > Rat License check is not effective > -- > >

[GitHub] [hudi] xushiyan opened a new pull request, #5273: [HUDI-3837] Fix license and rat check settings

2022-04-09 Thread GitBox
xushiyan opened a new pull request, #5273: URL: https://github.com/apache/hudi/pull/5273 - add missing licenses - fix CI setting to run rat plugin - fix deploy script to include integ test modules -- This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Updated] (HUDI-3837) Rat License check is not effective

2022-04-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3837: - Labels: pull-request-available (was: ) > Rat License check is not effective > ---

[GitHub] [hudi] hudi-bot commented on pull request #5273: [HUDI-3837] Fix license and rat check settings

2022-04-09 Thread GitBox
hudi-bot commented on PR #5273: URL: https://github.com/apache/hudi/pull/5273#issuecomment-1094077136 ## CI report: * d50b7d7b66ed9feb66d3169e075aa14b9fe002f5 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5273: [HUDI-3837] Fix license and rat check settings

2022-04-09 Thread GitBox
hudi-bot commented on PR #5273: URL: https://github.com/apache/hudi/pull/5273#issuecomment-1094077660 ## CI report: * d50b7d7b66ed9feb66d3169e075aa14b9fe002f5 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7946

[GitHub] [hudi] xushiyan commented on a diff in pull request #5273: [HUDI-3837] Fix license and rat check settings

2022-04-09 Thread GitBox
xushiyan commented on code in PR #5273: URL: https://github.com/apache/hudi/pull/5273#discussion_r846653463 ## pom.xml: ## @@ -1375,7 +1376,8 @@ integration-tests - integration-tests + deployArtifacts + true Review Comment:

[GitHub] [hudi] hudi-bot commented on pull request #5244: [HUDI-3812] Fixing Data Skipping configuration to respect Metadata Table configs

2022-04-09 Thread GitBox
hudi-bot commented on PR #5244: URL: https://github.com/apache/hudi/pull/5244#issuecomment-1094087138 ## CI report: * c252f8d6d9a6b38adcebec0ba857d5aafae823cf UNKNOWN * 6097520274d65ff9a9c2734a25eaf253ef78529d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] nsivabalan commented on a diff in pull request #5273: [HUDI-3837] Fix license and rat check settings

2022-04-09 Thread GitBox
nsivabalan commented on code in PR #5273: URL: https://github.com/apache/hudi/pull/5273#discussion_r846660586 ## pom.xml: ## @@ -1375,7 +1376,8 @@ integration-tests - integration-tests + deployArtifacts + true Review Comment:

[GitHub] [hudi] hudi-bot commented on pull request #5273: [HUDI-3837] Fix license and rat check settings

2022-04-09 Thread GitBox
hudi-bot commented on PR #5273: URL: https://github.com/apache/hudi/pull/5273#issuecomment-1094091029 ## CI report: * d50b7d7b66ed9feb66d3169e075aa14b9fe002f5 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7946

[GitHub] [hudi] xushiyan commented on a diff in pull request #5273: [HUDI-3837] Fix license and rat check settings

2022-04-09 Thread GitBox
xushiyan commented on code in PR #5273: URL: https://github.com/apache/hudi/pull/5273#discussion_r846661662 ## pom.xml: ## @@ -1375,7 +1376,8 @@ integration-tests - integration-tests + deployArtifacts + true Review Comment:

[GitHub] [hudi] hudi-bot commented on pull request #5266: [HUDI-3834] Fixing performance hits in reading Column Stats Index

2022-04-09 Thread GitBox
hudi-bot commented on PR #5266: URL: https://github.com/apache/hudi/pull/5266#issuecomment-1094092978 ## CI report: * 79639f70c86407d2cce694670fad2170e3f59ee7 UNKNOWN * 723c55a7638c2110494863da74d56da06dd261d0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #5266: [HUDI-3834] Fixing performance hits in reading Column Stats Index

2022-04-09 Thread GitBox
hudi-bot commented on PR #5266: URL: https://github.com/apache/hudi/pull/5266#issuecomment-1094094057 ## CI report: * 79639f70c86407d2cce694670fad2170e3f59ee7 UNKNOWN * 723c55a7638c2110494863da74d56da06dd261d0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #5244: [HUDI-3812] Fixing Data Skipping configuration to respect Metadata Table configs

2022-04-09 Thread GitBox
hudi-bot commented on PR #5244: URL: https://github.com/apache/hudi/pull/5244#issuecomment-1094094556 ## CI report: * c252f8d6d9a6b38adcebec0ba857d5aafae823cf UNKNOWN * 6097520274d65ff9a9c2734a25eaf253ef78529d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] nsivabalan commented on a diff in pull request #5273: [HUDI-3837] Fix license and rat check settings

2022-04-09 Thread GitBox
nsivabalan commented on code in PR #5273: URL: https://github.com/apache/hudi/pull/5273#discussion_r846662951 ## pom.xml: ## @@ -1375,7 +1376,8 @@ integration-tests - integration-tests + deployArtifacts + true Review Comment:

[GitHub] [hudi] xushiyan merged pull request #5273: [HUDI-3837] Fix license and rat check settings

2022-04-09 Thread GitBox
xushiyan merged PR #5273: URL: https://github.com/apache/hudi/pull/5273 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.

[hudi] branch master updated (81b25c543a -> 5e65aefc61)

2022-04-09 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 81b25c543a [HUDI-3825] Fixing Column Stats Index updating sequence (#5267) add 5e65aefc61 [HUDI-3837] Fix licens

[jira] [Closed] (HUDI-3837) Rat License check is not effective

2022-04-09 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-3837. Assignee: Raymond Xu Resolution: Fixed > Rat License check is not effective >

[jira] [Created] (HUDI-3838) Make Drop partition column config work with deltastreamer

2022-04-09 Thread Raymond Xu (Jira)
Raymond Xu created HUDI-3838: Summary: Make Drop partition column config work with deltastreamer Key: HUDI-3838 URL: https://issues.apache.org/jira/browse/HUDI-3838 Project: Apache Hudi Issue Typ

[jira] [Updated] (HUDI-3838) Make Drop partition column config work with deltastreamer

2022-04-09 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3838: - Description: hoodie.datasource.write.drop.partition.columns only works for datasource writer. HoodieDeltaS

[GitHub] [hudi] hudi-bot commented on pull request #5244: [HUDI-3812] Fixing Data Skipping configuration to respect Metadata Table configs

2022-04-09 Thread GitBox
hudi-bot commented on PR #5244: URL: https://github.com/apache/hudi/pull/5244#issuecomment-1094106725 ## CI report: * c252f8d6d9a6b38adcebec0ba857d5aafae823cf UNKNOWN * cf0ecea0b15e98a0d41f830c3b26e6b498d279d6 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] rahil-c commented on pull request #5250: [HUDI-3817] specify parquet version for hudi-hadoop-mr-bundle when compile hudi using -Dspark3

2022-04-09 Thread GitBox
rahil-c commented on PR #5250: URL: https://github.com/apache/hudi/pull/5250#issuecomment-1094107957 I think for my pr https://github.com/rahil-c/hudi/commits/rchertar/hdp-3-spark-3 I had discussed with @yihua the approach of defining parquet 0.10.x in hudi hadoop mr bundle because thats w

[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #5141: [HUDI-3724] Fixing closure of ParquetReader

2022-04-09 Thread GitBox
alexeykudinkin commented on code in PR #5141: URL: https://github.com/apache/hudi/pull/5141#discussion_r846670094 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieBaseRelation.scala: ## @@ -333,7 +335,13 @@ object HoodieBaseRelation { partitio

[GitHub] [hudi] hudi-bot commented on pull request #5266: [HUDI-3834] Fixing performance hits in reading Column Stats Index

2022-04-09 Thread GitBox
hudi-bot commented on PR #5266: URL: https://github.com/apache/hudi/pull/5266#issuecomment-1094108241 ## CI report: * 79639f70c86407d2cce694670fad2170e3f59ee7 UNKNOWN * a87c363fbd49ab8ce17a5f1d77d92205b7f9ada4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] nsivabalan merged pull request #5268: [HUDI-3807] Add a new config to control the use of metadata index in HoodieBloomIndex

2022-04-09 Thread GitBox
nsivabalan merged PR #5268: URL: https://github.com/apache/hudi/pull/5268 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apach

[hudi] branch master updated (5e65aefc61 -> 3e97c88c4f)

2022-04-09 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 5e65aefc61 [HUDI-3837] Fix license and rat check settings (#5273) add 3e97c88c4f [HUDI-3807] Add a new config to

[GitHub] [hudi] alexeykudinkin commented on pull request #5266: [HUDI-3834] Fixing performance hits in reading Column Stats Index

2022-04-09 Thread GitBox
alexeykudinkin commented on PR #5266: URL: https://github.com/apache/hudi/pull/5266#issuecomment-1094115388 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

[jira] [Created] (HUDI-3839) List of MT partitions to be updated is selected incorrectly

2022-04-09 Thread Alexey Kudinkin (Jira)
Alexey Kudinkin created HUDI-3839: - Summary: List of MT partitions to be updated is selected incorrectly Key: HUDI-3839 URL: https://issues.apache.org/jira/browse/HUDI-3839 Project: Apache Hudi

[GitHub] [hudi] hudi-bot commented on pull request #5266: [HUDI-3834] Fixing performance hits in reading Column Stats Index

2022-04-09 Thread GitBox
hudi-bot commented on PR #5266: URL: https://github.com/apache/hudi/pull/5266#issuecomment-1094115670 ## CI report: * 79639f70c86407d2cce694670fad2170e3f59ee7 UNKNOWN * a87c363fbd49ab8ce17a5f1d77d92205b7f9ada4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] alexeykudinkin opened a new pull request, #5274: [HUDI-3839] Fixing incorrect selection of MT partitions to be updated

2022-04-09 Thread GitBox
alexeykudinkin opened a new pull request, #5274: URL: https://github.com/apache/hudi/pull/5274 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

[jira] [Updated] (HUDI-3839) List of MT partitions to be updated is selected incorrectly

2022-04-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3839: - Labels: pull-request-available (was: ) > List of MT partitions to be updated is selected incorrec

[GitHub] [hudi] hudi-bot commented on pull request #5274: [HUDI-3839] Fixing incorrect selection of MT partitions to be updated

2022-04-09 Thread GitBox
hudi-bot commented on PR #5274: URL: https://github.com/apache/hudi/pull/5274#issuecomment-1094117209 ## CI report: * d71b0cb73dfff378755c5ffebdb9b01f1e0c33fb UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] nsivabalan commented on a diff in pull request #5244: [HUDI-3812] Fixing Data Skipping configuration to respect Metadata Table configs

2022-04-09 Thread GitBox
nsivabalan commented on code in PR #5244: URL: https://github.com/apache/hudi/pull/5244#discussion_r846676319 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieFileIndex.scala: ## @@ -196,12 +191,20 @@ case class HoodieFileIndex(spark: SparkSession,

[GitHub] [hudi] hudi-bot commented on pull request #5193: [DO NOT MERGE][DBG] Enabling Column Stats Index by default for all columns

2022-04-09 Thread GitBox
hudi-bot commented on PR #5193: URL: https://github.com/apache/hudi/pull/5193#issuecomment-1094117592 ## CI report: * b2028eefb54ce1c63f6653fb74a91b96e804baff Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7934

[GitHub] [hudi] hudi-bot commented on pull request #5274: [HUDI-3839] Fixing incorrect selection of MT partitions to be updated

2022-04-09 Thread GitBox
hudi-bot commented on PR #5274: URL: https://github.com/apache/hudi/pull/5274#issuecomment-1094117623 ## CI report: * d71b0cb73dfff378755c5ffebdb9b01f1e0c33fb Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7952

[GitHub] [hudi] hudi-bot commented on pull request #5193: [DO NOT MERGE][DBG] Enabling Column Stats Index by default for all columns

2022-04-09 Thread GitBox
hudi-bot commented on PR #5193: URL: https://github.com/apache/hudi/pull/5193#issuecomment-1094117999 ## CI report: * b2028eefb54ce1c63f6653fb74a91b96e804baff Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7934

[GitHub] [hudi] nsivabalan commented on a diff in pull request #5244: [HUDI-3812] Fixing Data Skipping configuration to respect Metadata Table configs

2022-04-09 Thread GitBox
nsivabalan commented on code in PR #5244: URL: https://github.com/apache/hudi/pull/5244#discussion_r846676628 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieFileIndex.scala: ## @@ -196,12 +191,20 @@ case class HoodieFileIndex(spark: SparkSession,

[GitHub] [hudi] nsivabalan commented on a diff in pull request #5274: [HUDI-3839] Fixing incorrect selection of MT partitions to be updated

2022-04-09 Thread GitBox
nsivabalan commented on code in PR #5274: URL: https://github.com/apache/hudi/pull/5274#discussion_r846676945 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -770,18 +774,6 @@ private MetadataRecordsGenerationP

[jira] [Updated] (HUDI-3840) Warn logs about not able to read replace commit metadata

2022-04-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3840: -- Fix Version/s: 0.12.0 > Warn logs about not able to read replace commit metadata >

[jira] [Created] (HUDI-3840) Warn logs about not able to read replace commit metadata

2022-04-09 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-3840: - Summary: Warn logs about not able to read replace commit metadata Key: HUDI-3840 URL: https://issues.apache.org/jira/browse/HUDI-3840 Project: Apache Hudi

[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #5244: [HUDI-3812] Fixing Data Skipping configuration to respect Metadata Table configs

2022-04-09 Thread GitBox
alexeykudinkin commented on code in PR #5244: URL: https://github.com/apache/hudi/pull/5244#discussion_r846678629 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieFileIndex.scala: ## @@ -196,12 +191,20 @@ case class HoodieFileIndex(spark: SparkSess

[GitHub] [hudi] hudi-bot commented on pull request #5266: [HUDI-3834] Fixing performance hits in reading Column Stats Index

2022-04-09 Thread GitBox
hudi-bot commented on PR #5266: URL: https://github.com/apache/hudi/pull/5266#issuecomment-1094125899 ## CI report: * 79639f70c86407d2cce694670fad2170e3f59ee7 UNKNOWN * a87c363fbd49ab8ce17a5f1d77d92205b7f9ada4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #5274: [HUDI-3839] Fixing incorrect selection of MT partitions to be updated

2022-04-09 Thread GitBox
alexeykudinkin commented on code in PR #5274: URL: https://github.com/apache/hudi/pull/5274#discussion_r846681793 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -770,18 +774,6 @@ private MetadataRecordsGenerat

[GitHub] [hudi] hudi-bot commented on pull request #5274: [HUDI-3839] Fixing incorrect selection of MT partitions to be updated

2022-04-09 Thread GitBox
hudi-bot commented on PR #5274: URL: https://github.com/apache/hudi/pull/5274#issuecomment-1094131521 ## CI report: * d71b0cb73dfff378755c5ffebdb9b01f1e0c33fb Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7952

[GitHub] [hudi] nsivabalan commented on a diff in pull request #5274: [HUDI-3839] Fixing incorrect selection of MT partitions to be updated

2022-04-09 Thread GitBox
nsivabalan commented on code in PR #5274: URL: https://github.com/apache/hudi/pull/5274#discussion_r846685721 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -770,18 +774,6 @@ private MetadataRecordsGenerationP

[jira] [Created] (HUDI-3841) Data Skipping is not working correctly in the presence of Schema Evolution

2022-04-09 Thread Alexey Kudinkin (Jira)
Alexey Kudinkin created HUDI-3841: - Summary: Data Skipping is not working correctly in the presence of Schema Evolution Key: HUDI-3841 URL: https://issues.apache.org/jira/browse/HUDI-3841 Project: Apa

[jira] [Updated] (HUDI-3839) List of MT partitions to be updated is selected incorrectly

2022-04-09 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3839: -- Status: Patch Available (was: In Progress) > List of MT partitions to be updated is selected in

[jira] [Updated] (HUDI-3839) List of MT partitions to be updated is selected incorrectly

2022-04-09 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3839: -- Sprint: Hudi-Sprint-Apr-05 > List of MT partitions to be updated is selected incorrectly > -

[jira] [Updated] (HUDI-3839) List of MT partitions to be updated is selected incorrectly

2022-04-09 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3839: -- Status: In Progress (was: Open) > List of MT partitions to be updated is selected incorrectly >

[jira] [Updated] (HUDI-3841) Data Skipping is not working correctly in the presence of Schema Evolution

2022-04-09 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3841: -- Sprint: Hudi-Sprint-Apr-05 > Data Skipping is not working correctly in the presence of Schema Ev

[jira] [Updated] (HUDI-3825) Fix tests failing when enabling MT on the Read Path

2022-04-09 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3825: -- Status: Patch Available (was: In Progress) > Fix tests failing when enabling MT on the Read Pat

[jira] [Updated] (HUDI-3841) Data Skipping is not working correctly in the presence of Schema Evolution

2022-04-09 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3841: -- Status: Patch Available (was: In Progress) > Data Skipping is not working correctly in the pres

[jira] [Updated] (HUDI-3841) Data Skipping is not working correctly in the presence of Schema Evolution

2022-04-09 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3841: -- Status: In Progress (was: Open) > Data Skipping is not working correctly in the presence of Sch

[GitHub] [hudi] alexeykudinkin opened a new pull request, #5275: [HUDI-3841][Stacked on 5244] Fixing Column Stats in the presence of Schema Evolution

2022-04-09 Thread GitBox
alexeykudinkin opened a new pull request, #5275: URL: https://github.com/apache/hudi/pull/5275 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

[jira] [Updated] (HUDI-3841) Data Skipping is not working correctly in the presence of Schema Evolution

2022-04-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3841: - Labels: pull-request-available (was: ) > Data Skipping is not working correctly in the presence o

[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #5244: [HUDI-3812] Fixing Data Skipping configuration to respect Metadata Table configs

2022-04-09 Thread GitBox
alexeykudinkin commented on code in PR #5244: URL: https://github.com/apache/hudi/pull/5244#discussion_r846686802 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieFileIndex.scala: ## @@ -196,12 +191,20 @@ case class HoodieFileIndex(spark: SparkSess

[GitHub] [hudi] hudi-bot commented on pull request #5275: [HUDI-3841][Stacked on 5244] Fixing Column Stats in the presence of Schema Evolution

2022-04-09 Thread GitBox
hudi-bot commented on PR #5275: URL: https://github.com/apache/hudi/pull/5275#issuecomment-1094134821 ## CI report: * 259cb0976de4dd8e9d2f9b690a77e00861d53ed5 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5193: [DO NOT MERGE][DBG] Enabling Column Stats Index by default for all columns

2022-04-09 Thread GitBox
hudi-bot commented on PR #5193: URL: https://github.com/apache/hudi/pull/5193#issuecomment-1094135140 ## CI report: * fe2607a1ff7a0d182e9dd18b3e2e40feeb3d35bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7953

[GitHub] [hudi] hudi-bot commented on pull request #5275: [HUDI-3841][Stacked on 5244] Fixing Column Stats in the presence of Schema Evolution

2022-04-09 Thread GitBox
hudi-bot commented on PR #5275: URL: https://github.com/apache/hudi/pull/5275#issuecomment-1094135164 ## CI report: * 259cb0976de4dd8e9d2f9b690a77e00861d53ed5 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7954

[GitHub] [hudi] hudi-bot commented on pull request #5275: [HUDI-3841][Stacked on 5244] Fixing Column Stats in the presence of Schema Evolution

2022-04-09 Thread GitBox
hudi-bot commented on PR #5275: URL: https://github.com/apache/hudi/pull/5275#issuecomment-1094142871 ## CI report: * 259cb0976de4dd8e9d2f9b690a77e00861d53ed5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7954

[jira] [Created] (HUDI-3842) Add non partitioned tests to integ tests

2022-04-09 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-3842: - Summary: Add non partitioned tests to integ tests Key: HUDI-3842 URL: https://issues.apache.org/jira/browse/HUDI-3842 Project: Apache Hudi Issue Ty

[jira] [Updated] (HUDI-3842) Add non partitioned tests to integ tests

2022-04-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3842: -- Fix Version/s: 0.12.0 > Add non partitioned tests to integ tests > -

[jira] [Updated] (HUDI-3842) Add non partitioned tests to integ tests

2022-04-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3842: -- Epic Link: HUDI-1236 Story Points: 2 > Add non partitioned tests to integ tests >

[jira] [Updated] (HUDI-3842) Add non partitioned tests to integ tests

2022-04-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3842: -- Priority: Blocker (was: Major) > Add non partitioned tests to integ tests > ---

[jira] [Assigned] (HUDI-3842) Add non partitioned tests to integ tests

2022-04-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-3842: - Assignee: sivabalan narayanan > Add non partitioned tests to integ tests > --

[GitHub] [hudi] nsivabalan opened a new pull request, #5276: [HUDI-3842] Integ tests for non partitioned datasets

2022-04-09 Thread GitBox
nsivabalan opened a new pull request, #5276: URL: https://github.com/apache/hudi/pull/5276 ## What is the purpose of the pull request - Adding support for testing non-partitioned datasets w/ integ tests - Added few test yamls and properties files - Fixed few other issues w/ integ

[jira] [Updated] (HUDI-3842) Add non partitioned tests to integ tests

2022-04-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3842: - Labels: pull-request-available (was: ) > Add non partitioned tests to integ tests > -

[GitHub] [hudi] hudi-bot commented on pull request #5276: [HUDI-3842] Integ tests for non partitioned datasets

2022-04-09 Thread GitBox
hudi-bot commented on PR #5276: URL: https://github.com/apache/hudi/pull/5276#issuecomment-1094144123 ## CI report: * 5e3c36b7670e8a23a455a1656d257f42f02ca848 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5276: [HUDI-3842] Integ tests for non partitioned datasets

2022-04-09 Thread GitBox
hudi-bot commented on PR #5276: URL: https://github.com/apache/hudi/pull/5276#issuecomment-1094144721 ## CI report: * 5e3c36b7670e8a23a455a1656d257f42f02ca848 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7955

[GitHub] [hudi] yihua merged pull request #5271: [MINOR] Fix typos in the comments of HoodieMergeHandle

2022-04-09 Thread GitBox
yihua merged PR #5271: URL: https://github.com/apache/hudi/pull/5271 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

[hudi] branch master updated (3e97c88c4f -> 15c264535f)

2022-04-09 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 3e97c88c4f [HUDI-3807] Add a new config to control the use of metadata index in HoodieBloomIndex (#5268) add 15c264

[GitHub] [hudi] moustafaalaa commented on pull request #4167: [HUDI-2880] Fixing loading of props from default dir

2022-04-09 Thread GitBox
moustafaalaa commented on PR #4167: URL: https://github.com/apache/hudi/pull/4167#issuecomment-1094153012 @nsivabalan @vinothchandar I tried to set the path of the file using Spark Config and Python Environment config and it doesn't work Spark Config: `pyspark` ``` spark =

[GitHub] [hudi] hudi-bot commented on pull request #5276: [HUDI-3842] Integ tests for non partitioned datasets

2022-04-09 Thread GitBox
hudi-bot commented on PR #5276: URL: https://github.com/apache/hudi/pull/5276#issuecomment-1094155140 ## CI report: * 5e3c36b7670e8a23a455a1656d257f42f02ca848 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7955

[GitHub] [hudi] whitecloud6688 opened a new issue, #5277: [SUPPORT] FlinkSQL中创建的表结构信息在SparkSQL中不可见

2022-04-09 Thread GitBox
whitecloud6688 opened a new issue, #5277: URL: https://github.com/apache/hudi/issues/5277 ### FlinkSQL中创建的表结构信息在SparkSQL中不可见 FlinkSQL中创建的表结构信息在SparkSQL中不可见,导致在SparkSQL中不能查询FlinkSQL定义的表数据。 **FlinkSQL中创建hudi表和mysql表,在FlinkSQL能够看到表结构信息。** [flink]$ bin/sql-client.sh Flink SQL> CRE

[GitHub] [hudi] whitecloud6688 opened a new issue, #5278: [SUPPORT] FlinkSQL中创建的表结构信息在SparkSQL中不可见

2022-04-09 Thread GitBox
whitecloud6688 opened a new issue, #5278: URL: https://github.com/apache/hudi/issues/5278 ### FlinkSQL中创建的表结构信息在SparkSQL中不可见 FlinkSQL中创建的表结构信息在SparkSQL中不可见、字段的注释信息不可见,导致在SparkSQL中不能查询FlinkSQL定义的表数据。 **FlinkSQL中创建hudi表和mysql表,在FlinkSQL能够看到表结构信息。** [flink]$ bin/sql-client.sh Fli

[GitHub] [hudi] hudi-bot commented on pull request #5272: [WIP][HUDI-3826] Commands deleting partitions do so incorrectly

2022-04-09 Thread GitBox
hudi-bot commented on PR #5272: URL: https://github.com/apache/hudi/pull/5272#issuecomment-1094169110 ## CI report: * 76a2f42dbb28b75940f3b8ff28a41d96dae8e4d0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7945

[GitHub] [hudi] hudi-bot commented on pull request #5272: [WIP][HUDI-3826] Commands deleting partitions do so incorrectly

2022-04-09 Thread GitBox
hudi-bot commented on PR #5272: URL: https://github.com/apache/hudi/pull/5272#issuecomment-1094169443 ## CI report: * 76a2f42dbb28b75940f3b8ff28a41d96dae8e4d0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7945

[jira] [Created] (HUDI-3843) Make Flink bundle only build with scala 2.11

2022-04-09 Thread Raymond Xu (Jira)
Raymond Xu created HUDI-3843: Summary: Make Flink bundle only build with scala 2.11 Key: HUDI-3843 URL: https://issues.apache.org/jira/browse/HUDI-3843 Project: Apache Hudi Issue Type: Improvemen

[jira] [Updated] (HUDI-3843) Make Flink 1.13.x 1.14.x build with scala 2.11

2022-04-09 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3843: - Summary: Make Flink 1.13.x 1.14.x build with scala 2.11 (was: Make Flink bundle only build with scala 2.1

[GitHub] [hudi] whitecloud6688 closed issue #5277: [SUPPORT] FlinkSQL中创建的表结构信息在SparkSQL中不可见

2022-04-09 Thread GitBox
whitecloud6688 closed issue #5277: [SUPPORT] FlinkSQL中创建的表结构信息在SparkSQL中不可见 URL: https://github.com/apache/hudi/issues/5277 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

[GitHub] [hudi] whitecloud6688 commented on issue #5277: [SUPPORT] FlinkSQL中创建的表结构信息在SparkSQL中不可见

2022-04-09 Thread GitBox
whitecloud6688 commented on issue #5277: URL: https://github.com/apache/hudi/issues/5277#issuecomment-1094170714 网速太慢,重复提交 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [hudi] leesf commented on issue #5277: [SUPPORT] FlinkSQL中创建的表结构信息在SparkSQL中不可见

2022-04-09 Thread GitBox
leesf commented on issue #5277: URL: https://github.com/apache/hudi/issues/5277#issuecomment-1094170779 您好,您的邮件我已经收到,我会立即回复,祝您生活愉快! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [hudi] xushiyan opened a new pull request, #5279: [HUDI-3843] Make flink profiles build with scala-2.11

2022-04-09 Thread GitBox
xushiyan opened a new pull request, #5279: URL: https://github.com/apache/hudi/pull/5279 - Make flink 1.13 and 1.14 profiles build with scala 2.11 during deploy - Update GH actions setup accordingly - Update README -- This is an automated message from the Apache Git Service. To resp

[jira] [Updated] (HUDI-3843) Make Flink 1.13.x 1.14.x build with scala 2.11

2022-04-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3843: - Labels: pull-request-available (was: ) > Make Flink 1.13.x 1.14.x build with scala 2.11 > ---

[GitHub] [hudi] hudi-bot commented on pull request #5279: [HUDI-3843] Make flink profiles build with scala-2.11

2022-04-09 Thread GitBox
hudi-bot commented on PR #5279: URL: https://github.com/apache/hudi/pull/5279#issuecomment-1094171690 ## CI report: * b1d969ad49aa2978223c6a461c04a7e00450bb6b UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] xushiyan commented on a diff in pull request #5279: [HUDI-3843] Make flink profiles build with scala-2.11

2022-04-09 Thread GitBox
xushiyan commented on code in PR #5279: URL: https://github.com/apache/hudi/pull/5279#discussion_r846709506 ## scripts/release/deploy_staging_jars.sh: ## @@ -38,7 +38,8 @@ fi declare -a ALL_VERSION_OPTS=( "-Dscala-2.11 -Dspark2.4 -Dflink1.13" -"-Dscala-2.12 -Dspark2.4 -Dflin

  1   2   >