[jira] [Created] (HUDI-3844) HoodieIndexer should set existing MDT partitions in props if not already set by user

2022-04-10 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-3844: - Summary: HoodieIndexer should set existing MDT partitions in props if not already set by user Key: HUDI-3844 URL: https://issues.apache.org/jira/browse/HUDI-3844 Project: A

[jira] [Updated] (HUDI-3844) HoodieIndexer should set existing MDT partitions in props if not already set by user

2022-04-10 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3844: -- Fix Version/s: 0.11.0 > HoodieIndexer should set existing MDT partitions in props if not already set >

[GitHub] [hudi] ksoullpwk opened a new issue, #5281: [SUPPORT] .hoodie/hoodie.properties file can be deleted due to retention settings of cloud providers

2022-04-10 Thread GitBox
ksoullpwk opened a new issue, #5281: URL: https://github.com/apache/hudi/issues/5281 **Describe the problem you faced** `.hoodie/hoodie.properties` file can be deleted due to retention settings of cloud providers. Is there any configs we can set to refresh this properties file?

[GitHub] [hudi] wxplovecc commented on pull request #5066: [MINOR]flink ParquetSchemaConverter timestamp(3) should converted as TIMESTAMP_MILLIS for bulk_insert

2022-04-10 Thread GitBox
wxplovecc commented on PR #5066: URL: https://github.com/apache/hudi/pull/5066#issuecomment-1094231131 Fixed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [hudi] wxplovecc closed pull request #5066: [MINOR]flink ParquetSchemaConverter timestamp(3) should converted as TIMESTAMP_MILLIS for bulk_insert

2022-04-10 Thread GitBox
wxplovecc closed pull request #5066: [MINOR]flink ParquetSchemaConverter timestamp(3) should converted as TIMESTAMP_MILLIS for bulk_insert URL: https://github.com/apache/hudi/pull/5066 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [hudi] hudi-bot commented on pull request #5279: [HUDI-3843] Make flink profiles build with scala-2.11

2022-04-10 Thread GitBox
hudi-bot commented on PR #5279: URL: https://github.com/apache/hudi/pull/5279#issuecomment-1094240176 ## CI report: * b1d969ad49aa2978223c6a461c04a7e00450bb6b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7958

[GitHub] [hudi] hudi-bot commented on pull request #5279: [HUDI-3843] Make flink profiles build with scala-2.11

2022-04-10 Thread GitBox
hudi-bot commented on PR #5279: URL: https://github.com/apache/hudi/pull/5279#issuecomment-1094240589 ## CI report: * b1d969ad49aa2978223c6a461c04a7e00450bb6b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7958

[GitHub] [hudi] XuQianJin-Stars commented on pull request #5272: [HUDI-3826] Commands deleting partitions do so incorrectly

2022-04-10 Thread GitBox
XuQianJin-Stars commented on PR #5272: URL: https://github.com/apache/hudi/pull/5272#issuecomment-1094241168 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [hudi] hudi-bot commented on pull request #5272: [HUDI-3826] Commands deleting partitions do so incorrectly

2022-04-10 Thread GitBox
hudi-bot commented on PR #5272: URL: https://github.com/apache/hudi/pull/5272#issuecomment-1094241558 ## CI report: * 1cb81a90f3053bb03de07c384771723ebaa80cce Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7957

[GitHub] [hudi] hudi-bot commented on pull request #5279: [HUDI-3843] Make flink profiles build with scala-2.11

2022-04-10 Thread GitBox
hudi-bot commented on PR #5279: URL: https://github.com/apache/hudi/pull/5279#issuecomment-1094244392 ## CI report: * b1d969ad49aa2978223c6a461c04a7e00450bb6b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7958

[GitHub] [hudi] hudi-bot commented on pull request #5279: [HUDI-3843] Make flink profiles build with scala-2.11

2022-04-10 Thread GitBox
hudi-bot commented on PR #5279: URL: https://github.com/apache/hudi/pull/5279#issuecomment-1094244878 ## CI report: * e2a37767f4540ca42c6a5eed4842acff75b6423d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=795

[GitHub] [hudi] hudi-bot commented on pull request #5279: [HUDI-3843] Make flink profiles build with scala-2.11

2022-04-10 Thread GitBox
hudi-bot commented on PR #5279: URL: https://github.com/apache/hudi/pull/5279#issuecomment-1094245515 ## CI report: * e2a37767f4540ca42c6a5eed4842acff75b6423d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=795

[GitHub] [hudi] hudi-bot commented on pull request #5279: [HUDI-3843] Make flink profiles build with scala-2.11

2022-04-10 Thread GitBox
hudi-bot commented on PR #5279: URL: https://github.com/apache/hudi/pull/5279#issuecomment-1094246156 ## CI report: * e2a37767f4540ca42c6a5eed4842acff75b6423d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=795

[GitHub] [hudi] hudi-bot commented on pull request #5279: [HUDI-3843] Make flink profiles build with scala-2.11

2022-04-10 Thread GitBox
hudi-bot commented on PR #5279: URL: https://github.com/apache/hudi/pull/5279#issuecomment-1094251611 ## CI report: * e2a37767f4540ca42c6a5eed4842acff75b6423d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=795

[jira] [Created] (HUDI-3845) Fix delete mor table's partition with urlencode's error

2022-04-10 Thread Forward Xu (Jira)
Forward Xu created HUDI-3845: Summary: Fix delete mor table's partition with urlencode's error Key: HUDI-3845 URL: https://issues.apache.org/jira/browse/HUDI-3845 Project: Apache Hudi Issue Type:

[GitHub] [hudi] XuQianJin-Stars opened a new pull request, #5282: [HUDI-3845] Fix delete mor table's partition with urlencode's error

2022-04-10 Thread GitBox
XuQianJin-Stars opened a new pull request, #5282: URL: https://github.com/apache/hudi/pull/5282 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

[jira] [Updated] (HUDI-3845) Fix delete mor table's partition with urlencode's error

2022-04-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3845: - Labels: pull-request-available (was: ) > Fix delete mor table's partition with urlencode's error

[jira] [Updated] (HUDI-3845) Fix delete mor table's partition with urlencode's error

2022-04-10 Thread Forward Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Forward Xu updated HUDI-3845: - Description: {code:java} // code placeholder 4604 [ScalaTest-run-running-TestDeleteTable] WARN   org.apach

[jira] [Updated] (HUDI-3845) Fix delete mor table's partition with urlencode's error

2022-04-10 Thread Forward Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Forward Xu updated HUDI-3845: - Description: {code:java} // code placeholder 4604 [ScalaTest-run-running-TestDeleteTable] WARN   org.apach

[GitHub] [hudi] hudi-bot commented on pull request #5282: [HUDI-3845] Fix delete mor table's partition with urlencode's error

2022-04-10 Thread GitBox
hudi-bot commented on PR #5282: URL: https://github.com/apache/hudi/pull/5282#issuecomment-1094255054 ## CI report: * ec64e73e3e0fd7e50d3c0057875d49aa99f91a5e UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5282: [HUDI-3845] Fix delete mor table's partition with urlencode's error

2022-04-10 Thread GitBox
hudi-bot commented on PR #5282: URL: https://github.com/apache/hudi/pull/5282#issuecomment-1094255476 ## CI report: * ec64e73e3e0fd7e50d3c0057875d49aa99f91a5e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7962

[GitHub] [hudi] hudi-bot commented on pull request #5272: [HUDI-3826] Commands deleting partitions do so incorrectly

2022-04-10 Thread GitBox
hudi-bot commented on PR #5272: URL: https://github.com/apache/hudi/pull/5272#issuecomment-1094256007 ## CI report: * 1cb81a90f3053bb03de07c384771723ebaa80cce Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7957

[GitHub] [hudi] xushiyan commented on a diff in pull request #5282: [HUDI-3845] Fix delete mor table's partition with urlencode's error

2022-04-10 Thread GitBox
xushiyan commented on code in PR #5282: URL: https://github.com/apache/hudi/pull/5282#discussion_r846771115 ## hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java: ## @@ -471,7 +470,8 @@ public void remove() { private static FSDataInputStream g

[GitHub] [hudi] hudi-bot commented on pull request #5279: [HUDI-3843] Make flink profiles build with scala-2.11

2022-04-10 Thread GitBox
hudi-bot commented on PR #5279: URL: https://github.com/apache/hudi/pull/5279#issuecomment-1094266855 ## CI report: * bac78a46dc8aac61b65c313ab399628ac39609df UNKNOWN * 9354a40ffdd90b98bdf29d2e7f0abc9e63f08edb UNKNOWN * 0a1f252a0f37753e340734c4f22bbe2f085ed461 Azure: [FAILUR

[GitHub] [hudi] hudi-bot commented on pull request #5279: [HUDI-3843] Make flink profiles build with scala-2.11

2022-04-10 Thread GitBox
hudi-bot commented on PR #5279: URL: https://github.com/apache/hudi/pull/5279#issuecomment-1094267620 ## CI report: * 0a1f252a0f37753e340734c4f22bbe2f085ed461 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5279: [HUDI-3843] Make flink profiles build with scala-2.11

2022-04-10 Thread GitBox
hudi-bot commented on PR #5279: URL: https://github.com/apache/hudi/pull/5279#issuecomment-1094268114 ## CI report: * 0a1f252a0f37753e340734c4f22bbe2f085ed461 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7961

[GitHub] [hudi] hudi-bot commented on pull request #5282: [HUDI-3845] Fix delete mor table's partition with urlencode's error

2022-04-10 Thread GitBox
hudi-bot commented on PR #5282: URL: https://github.com/apache/hudi/pull/5282#issuecomment-1094274177 ## CI report: * ec64e73e3e0fd7e50d3c0057875d49aa99f91a5e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7962

[GitHub] [hudi] hudi-bot commented on pull request #5279: [HUDI-3843] Make flink profiles build with scala-2.11

2022-04-10 Thread GitBox
hudi-bot commented on PR #5279: URL: https://github.com/apache/hudi/pull/5279#issuecomment-1094282727 ## CI report: * 0a1f252a0f37753e340734c4f22bbe2f085ed461 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7961

[GitHub] [hudi] vinothchandar commented on a diff in pull request #5269: [HUDI-3636] Create new write clients for async table services in DeltaStreamer

2022-04-10 Thread GitBox
vinothchandar commented on code in PR #5269: URL: https://github.com/apache/hudi/pull/5269#discussion_r846787619 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseCompactor.java: ## @@ -31,16 +33,30 @@ private static final long serialVersionUID = 1L

[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #5274: [HUDI-3839] Fixing incorrect selection of MT partitions to be updated

2022-04-10 Thread GitBox
alexeykudinkin commented on code in PR #5274: URL: https://github.com/apache/hudi/pull/5274#discussion_r846804960 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -770,18 +774,6 @@ private MetadataRecordsGenerat

[GitHub] [hudi] nsivabalan merged pull request #5266: [HUDI-3834] Fixing performance hits in reading Column Stats Index

2022-04-10 Thread GitBox
nsivabalan merged PR #5266: URL: https://github.com/apache/hudi/pull/5266 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apach

[hudi] branch master updated (15c264535f -> 7a9d48d126)

2022-04-10 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 15c264535f [MINOR] Fix typos in the comments of HoodieMergeHandle (#5271) add 7a9d48d126 [HUDI-3834] Fixing per

[GitHub] [hudi] nsivabalan merged pull request #5244: [HUDI-3812] Fixing Data Skipping configuration to respect Metadata Table configs

2022-04-10 Thread GitBox
nsivabalan merged PR #5244: URL: https://github.com/apache/hudi/pull/5244 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apach

[hudi] branch master updated: [HUDI-3812] Fixing Data Skipping configuration to respect Metadata Table configs (#5244)

2022-04-10 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 976840e8eb [HUDI-3812] Fixing Data Skipping con

[jira] [Commented] (HUDI-2524) Certify Hive sync on cloud platforms

2022-04-10 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17520210#comment-17520210 ] Raymond Xu commented on HUDI-2524: -- Previous failure was due to not selecting "Use for Sp

[jira] [Closed] (HUDI-2524) Certify Hive sync on cloud platforms

2022-04-10 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-2524. Resolution: Done > Certify Hive sync on cloud platforms > > >

[GitHub] [hudi] hudi-bot commented on pull request #5276: [HUDI-3842] Integ tests for non partitioned datasets

2022-04-10 Thread GitBox
hudi-bot commented on PR #5276: URL: https://github.com/apache/hudi/pull/5276#issuecomment-1094338305 ## CI report: * 5e3c36b7670e8a23a455a1656d257f42f02ca848 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7955

[GitHub] [hudi] hudi-bot commented on pull request #5276: [HUDI-3842] Integ tests for non partitioned datasets

2022-04-10 Thread GitBox
hudi-bot commented on PR #5276: URL: https://github.com/apache/hudi/pull/5276#issuecomment-1094338808 ## CI report: * 5e3c36b7670e8a23a455a1656d257f42f02ca848 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7955

[GitHub] [hudi] rahil-c opened a new pull request, #5283: Define Parquet version as 1.10.1 in Hudi Hadoop Mr

2022-04-10 Thread GitBox
rahil-c opened a new pull request, #5283: URL: https://github.com/apache/hudi/pull/5283 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpose

[GitHub] [hudi] hudi-bot commented on pull request #5283: Define Parquet version as 1.10.1 in Hudi Hadoop Mr

2022-04-10 Thread GitBox
hudi-bot commented on PR #5283: URL: https://github.com/apache/hudi/pull/5283#issuecomment-1094345192 ## CI report: * 531370598a08bcd8d161596ae8ee4833ae583886 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5283: Define Parquet version as 1.10.1 in Hudi Hadoop Mr

2022-04-10 Thread GitBox
hudi-bot commented on PR #5283: URL: https://github.com/apache/hudi/pull/5283#issuecomment-1094345596 ## CI report: * 531370598a08bcd8d161596ae8ee4833ae583886 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7965

[GitHub] [hudi] hudi-bot commented on pull request #5276: [HUDI-3842] Integ tests for non partitioned datasets

2022-04-10 Thread GitBox
hudi-bot commented on PR #5276: URL: https://github.com/apache/hudi/pull/5276#issuecomment-1094352962 ## CI report: * eea07419e482cabbc68a84f2c84cc3d81b61da42 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7964

[jira] [Resolved] (HUDI-3841) Data Skipping is not working correctly in the presence of Schema Evolution

2022-04-10 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin resolved HUDI-3841. --- > Data Skipping is not working correctly in the presence of Schema Evolution > ---

[jira] [Reopened] (HUDI-3841) Data Skipping is not working correctly in the presence of Schema Evolution

2022-04-10 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin reopened HUDI-3841: --- > Data Skipping is not working correctly in the presence of Schema Evolution > ---

[jira] [Resolved] (HUDI-3834) Evaluate MT Column Stats Performance

2022-04-10 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin resolved HUDI-3834. --- > Evaluate MT Column Stats Performance > - > >

[jira] [Closed] (HUDI-3812) Make sure Data Skipping respects Metadata Table config

2022-04-10 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin closed HUDI-3812. - Resolution: Fixed > Make sure Data Skipping respects Metadata Table config > -

[jira] [Closed] (HUDI-3834) Evaluate MT Column Stats Performance

2022-04-10 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin closed HUDI-3834. - Resolution: Fixed > Evaluate MT Column Stats Performance > -

[jira] [Closed] (HUDI-3841) Data Skipping is not working correctly in the presence of Schema Evolution

2022-04-10 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin closed HUDI-3841. - Resolution: Fixed > Data Skipping is not working correctly in the presence of Schema Evolution > -

[jira] [Updated] (HUDI-3841) Data Skipping is not working correctly in the presence of Schema Evolution

2022-04-10 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3841: -- Status: Patch Available (was: In Progress) > Data Skipping is not working correctly in the pres

[jira] [Closed] (HUDI-3825) Fix tests failing when enabling MT on the Read Path

2022-04-10 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin closed HUDI-3825. - Resolution: Fixed > Fix tests failing when enabling MT on the Read Path >

[GitHub] [hudi] hudi-bot commented on pull request #5283: Define Parquet version as 1.10.1 in Hudi Hadoop Mr

2022-04-10 Thread GitBox
hudi-bot commented on PR #5283: URL: https://github.com/apache/hudi/pull/5283#issuecomment-1094361870 ## CI report: * 531370598a08bcd8d161596ae8ee4833ae583886 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7965

[GitHub] [hudi] hudi-bot commented on pull request #5275: [HUDI-3841] Fixing Column Stats in the presence of Schema Evolution

2022-04-10 Thread GitBox
hudi-bot commented on PR #5275: URL: https://github.com/apache/hudi/pull/5275#issuecomment-1094363182 ## CI report: * 259cb0976de4dd8e9d2f9b690a77e00861d53ed5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7954

[GitHub] [hudi] hudi-bot commented on pull request #5193: [DO NOT MERGE][DBG] Enabling Column Stats Index by default for all columns

2022-04-10 Thread GitBox
hudi-bot commented on PR #5193: URL: https://github.com/apache/hudi/pull/5193#issuecomment-1094363603 ## CI report: * fe2607a1ff7a0d182e9dd18b3e2e40feeb3d35bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7953

[GitHub] [hudi] hudi-bot commented on pull request #5275: [HUDI-3841] Fixing Column Stats in the presence of Schema Evolution

2022-04-10 Thread GitBox
hudi-bot commented on PR #5275: URL: https://github.com/apache/hudi/pull/5275#issuecomment-1094363643 ## CI report: * 259cb0976de4dd8e9d2f9b690a77e00861d53ed5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7954

[GitHub] [hudi] hudi-bot commented on pull request #5193: [DO NOT MERGE][DBG] Enabling Column Stats Index by default for all columns

2022-04-10 Thread GitBox
hudi-bot commented on PR #5193: URL: https://github.com/apache/hudi/pull/5193#issuecomment-1094364195 ## CI report: * fe2607a1ff7a0d182e9dd18b3e2e40feeb3d35bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7953

[GitHub] [hudi] hudi-bot commented on pull request #5275: [HUDI-3841] Fixing Column Stats in the presence of Schema Evolution

2022-04-10 Thread GitBox
hudi-bot commented on PR #5275: URL: https://github.com/apache/hudi/pull/5275#issuecomment-1094377115 ## CI report: * f5340cc8ce15710625241a61cab8328fcc6a Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7966

[GitHub] [hudi] hudi-bot commented on pull request #5193: [DO NOT MERGE][DBG] Enabling Column Stats Index by default for all columns

2022-04-10 Thread GitBox
hudi-bot commented on PR #5193: URL: https://github.com/apache/hudi/pull/5193#issuecomment-1094378665 ## CI report: * b4211707fbb8d9adf5b777a3fad7906ee2dc9fc9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7967

[jira] [Created] (HUDI-3846) Clustering with allow update strategy could fail w/ async clustering

2022-04-10 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-3846: - Summary: Clustering with allow update strategy could fail w/ async clustering Key: HUDI-3846 URL: https://issues.apache.org/jira/browse/HUDI-3846 Project: A

[jira] [Updated] (HUDI-3846) Clustering with allow update strategy could fail w/ async clustering

2022-04-10 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3846: -- Fix Version/s: 0.12.0 > Clustering with allow update strategy could fail w/ async cluste

[GitHub] [hudi] yihua commented on a diff in pull request #5279: [HUDI-3843] Make flink profiles build with scala-2.11

2022-04-10 Thread GitBox
yihua commented on code in PR #5279: URL: https://github.com/apache/hudi/pull/5279#discussion_r846860962 ## scripts/release/deploy_staging_jars.sh: ## @@ -38,6 +38,7 @@ fi declare -a ALL_VERSION_OPTS=( "-Dscala-2.11 -Dspark2.4 -Dflink1.13" +"-Dscala-2.11 -Dspark2.4 -Dflink1.

[jira] [Closed] (HUDI-3807) Add separate configs to control multi-modal index usage in write-side indexing

2022-04-10 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo closed HUDI-3807. --- Resolution: Fixed > Add separate configs to control multi-modal index usage in write-side > indexing > -

[jira] [Created] (HUDI-3847) Fix NPE due to null schema in HoodieMetadataTableValidator

2022-04-10 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-3847: --- Summary: Fix NPE due to null schema in HoodieMetadataTableValidator Key: HUDI-3847 URL: https://issues.apache.org/jira/browse/HUDI-3847 Project: Apache Hudi Issue Type

[jira] [Updated] (HUDI-3847) Fix NPE due to null schema in HoodieMetadataTableValidator

2022-04-10 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3847: Sprint: Hudi-Sprint-Apr-05 > Fix NPE due to null schema in HoodieMetadataTableValidator > --

[GitHub] [hudi] yihua opened a new pull request, #5284: [HUDI-3847] Fix NPE due to null schema in HoodieMetadataTableValidator

2022-04-10 Thread GitBox
yihua opened a new pull request, #5284: URL: https://github.com/apache/hudi/pull/5284 ## What is the purpose of the pull request This PR fixes the NullPointerException during reading Hudi log files in HoodieMetadataTableValidator, possibly due to empty or partial log files written by

[jira] [Updated] (HUDI-3847) Fix NPE due to null schema in HoodieMetadataTableValidator

2022-04-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3847: - Labels: pull-request-available (was: ) > Fix NPE due to null schema in HoodieMetadataTableValidat

[GitHub] [hudi] hudi-bot commented on pull request #5284: [HUDI-3847] Fix NPE due to null schema in HoodieMetadataTableValidator

2022-04-10 Thread GitBox
hudi-bot commented on PR #5284: URL: https://github.com/apache/hudi/pull/5284#issuecomment-1094409556 ## CI report: * e99f79ba26ca5ed244e8819989fc8ed93f55f094 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5284: [HUDI-3847] Fix NPE due to null schema in HoodieMetadataTableValidator

2022-04-10 Thread GitBox
hudi-bot commented on PR #5284: URL: https://github.com/apache/hudi/pull/5284#issuecomment-1094410133 ## CI report: * e99f79ba26ca5ed244e8819989fc8ed93f55f094 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7968

[jira] [Updated] (HUDI-3847) Fix NPE due to null schema in HoodieMetadataTableValidator

2022-04-10 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3847: Issue Type: Bug (was: Task) > Fix NPE due to null schema in HoodieMetadataTableValidator >

[GitHub] [hudi] hudi-bot commented on pull request #5255: [HUDI-3798] Fixing ending of a transaction by different owner and removing some extraneous methods in trxn manager

2022-04-10 Thread GitBox
hudi-bot commented on PR #5255: URL: https://github.com/apache/hudi/pull/5255#issuecomment-1094415701 ## CI report: * d806482d7ea926a08b145a9d5983e68e56bc0bcf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7899

[GitHub] [hudi] hudi-bot commented on pull request #5255: [HUDI-3798] Fixing ending of a transaction by different owner and removing some extraneous methods in trxn manager

2022-04-10 Thread GitBox
hudi-bot commented on PR #5255: URL: https://github.com/apache/hudi/pull/5255#issuecomment-1094416454 ## CI report: * d806482d7ea926a08b145a9d5983e68e56bc0bcf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7899

[GitHub] [hudi] nsivabalan commented on a diff in pull request #5255: [HUDI-3798] Fixing ending of a transaction by different owner and removing some extraneous methods in trxn manager

2022-04-10 Thread GitBox
nsivabalan commented on code in PR #5255: URL: https://github.com/apache/hudi/pull/5255#discussion_r846870445 ## hudi-client/hudi-client-common/src/test/java/org/apache/hudi/client/transaction/TestTransactionManager.java: ## @@ -152,6 +165,32 @@ public void testMultiWriterTransa

[GitHub] [hudi] nsivabalan merged pull request #5276: [HUDI-3842] Integ tests for non partitioned datasets

2022-04-10 Thread GitBox
nsivabalan merged PR #5276: URL: https://github.com/apache/hudi/pull/5276 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apach

[hudi] branch master updated (976840e8eb -> 12731f5b89)

2022-04-10 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 976840e8eb [HUDI-3812] Fixing Data Skipping configuration to respect Metadata Table configs (#5244) add 12731f5

[jira] [Updated] (HUDI-3005) flink write hudi throw Not an Avro data file exception

2022-04-10 Thread waywtdcc (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc updated HUDI-3005: --- Fix Version/s: 0.9.0 > flink write hudi throw Not an Avro data file exception > --

[jira] [Updated] (HUDI-3005) flink write hudi throw Not an Avro data file exception

2022-04-10 Thread waywtdcc (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc updated HUDI-3005: --- Fix Version/s: (was: 0.9.0) > flink write hudi throw Not an Avro data file exception > ---

[jira] [Updated] (HUDI-3005) flink write hudi throw Not an Avro data file exception

2022-04-10 Thread waywtdcc (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] waywtdcc updated HUDI-3005: --- Fix Version/s: 0.9.0 > flink write hudi throw Not an Avro data file exception > --

[GitHub] [hudi] hudi-bot commented on pull request #5284: [HUDI-3847] Fix NPE due to null schema in HoodieMetadataTableValidator

2022-04-10 Thread GitBox
hudi-bot commented on PR #5284: URL: https://github.com/apache/hudi/pull/5284#issuecomment-1094432319 ## CI report: * e99f79ba26ca5ed244e8819989fc8ed93f55f094 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7968

[GitHub] [hudi] yihua commented on a diff in pull request #5261: [HUDI-3799] Fixing not deleting empty instants w/o archiving

2022-04-10 Thread GitBox
yihua commented on code in PR #5261: URL: https://github.com/apache/hudi/pull/5261#discussion_r846879376 ## hudi-common/src/test/java/org/apache/hudi/common/testutils/HoodieTestTable.java: ## @@ -286,8 +286,8 @@ public HoodieTestTable addClean(String instantTime, HoodieCleanerP

[hudi] branch master updated (12731f5b89 -> 63a099c5b7)

2022-04-10 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 12731f5b89 [HUDI-3842] Integ tests for non partitioned datasets (#5276) add 63a099c5b7 [HUDI-3847] Fix NPE due to nu

[GitHub] [hudi] yihua merged pull request #5284: [HUDI-3847] Fix NPE due to null schema in HoodieMetadataTableValidator

2022-04-10 Thread GitBox
yihua merged PR #5284: URL: https://github.com/apache/hudi/pull/5284 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

[jira] [Updated] (HUDI-3847) Fix NPE due to null schema in HoodieMetadataTableValidator

2022-04-10 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3847: Status: Patch Available (was: In Progress) > Fix NPE due to null schema in HoodieMetadataTableValidator > -

[jira] [Closed] (HUDI-3847) Fix NPE due to null schema in HoodieMetadataTableValidator

2022-04-10 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo closed HUDI-3847. --- Resolution: Fixed > Fix NPE due to null schema in HoodieMetadataTableValidator > -

[jira] [Updated] (HUDI-3847) Fix NPE due to null schema in HoodieMetadataTableValidator

2022-04-10 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3847: Status: In Progress (was: Open) > Fix NPE due to null schema in HoodieMetadataTableValidator >

[GitHub] [hudi] hudi-bot commented on pull request #5255: [HUDI-3798] Fixing ending of a transaction by different owner and removing some extraneous methods in trxn manager

2022-04-10 Thread GitBox
hudi-bot commented on PR #5255: URL: https://github.com/apache/hudi/pull/5255#issuecomment-1094443512 ## CI report: * 4af6b18971170b74c0a4bab43cf4233713e3f8c1 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7969

[GitHub] [hudi] nsivabalan commented on a diff in pull request #5261: [HUDI-3799] Fixing not deleting empty instants w/o archiving

2022-04-10 Thread GitBox
nsivabalan commented on code in PR #5261: URL: https://github.com/apache/hudi/pull/5261#discussion_r846900520 ## hudi-common/src/test/java/org/apache/hudi/common/testutils/HoodieTestTable.java: ## @@ -286,8 +286,8 @@ public HoodieTestTable addClean(String instantTime, HoodieCle

[GitHub] [hudi] nsivabalan commented on a diff in pull request #5261: [HUDI-3799] Fixing not deleting empty instants w/o archiving

2022-04-10 Thread GitBox
nsivabalan commented on code in PR #5261: URL: https://github.com/apache/hudi/pull/5261#discussion_r846900577 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/HoodieTimelineArchiver.java: ## @@ -632,8 +629,9 @@ private void writeToFile(Schema wrapperSchema,

[GitHub] [hudi] KnightChess commented on issue #5278: [SUPPORT] FlinkSQL中创建的表结构信息在SparkSQL中不可见

2022-04-10 Thread GitBox
KnightChess commented on issue #5278: URL: https://github.com/apache/hudi/issues/5278#issuecomment-1094477406 I think you can check is any table info in hive metastore, may be flink not sync meta to metastore. In spark 3.1.2, the "desc" meta data only from metastore. And flink create sql on

[GitHub] [hudi] huberylee opened a new pull request, #5285: Adding doc for procedures

2022-04-10 Thread GitBox
huberylee opened a new pull request, #5285: URL: https://github.com/apache/hudi/pull/5285 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpo

[jira] [Updated] (HUDI-3848) Restore fails when files pertaining to a commit has been cleaned up

2022-04-10 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3848: -- Fix Version/s: 0.11.0 > Restore fails when files pertaining to a commit has been cleaned

[jira] [Updated] (HUDI-3848) Restore fails when files pertaining to a commit has been cleaned up

2022-04-10 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3848: -- Priority: Blocker (was: Major) > Restore fails when files pertaining to a commit has be

[jira] [Assigned] (HUDI-3848) Restore fails when files pertaining to a commit has been cleaned up

2022-04-10 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-3848: - Assignee: sivabalan narayanan > Restore fails when files pertaining to a commit h

[jira] [Created] (HUDI-3848) Restore fails when files pertaining to a commit has been cleaned up

2022-04-10 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-3848: - Summary: Restore fails when files pertaining to a commit has been cleaned up Key: HUDI-3848 URL: https://issues.apache.org/jira/browse/HUDI-3848 Project: Ap

[jira] [Updated] (HUDI-3848) Restore fails when files pertaining to a commit has been cleaned up

2022-04-10 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3848: -- Description: DC1, DC2, savepoint DC2, DC3, C4 (compaction), DC5, DC6, C7 (compaction). 

[GitHub] [hudi] nsivabalan commented on a diff in pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

2022-04-10 Thread GitBox
nsivabalan commented on code in PR #4957: URL: https://github.com/apache/hudi/pull/4957#discussion_r846926292 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/rollback/ListingBasedRollbackStrategy.java: ## @@ -57,20 +83,260 @@ public ListingBasedRollb

[GitHub] [hudi] nsivabalan commented on a diff in pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

2022-04-10 Thread GitBox
nsivabalan commented on code in PR #4957: URL: https://github.com/apache/hudi/pull/4957#discussion_r846932161 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/rollback/ListingBasedRollbackStrategy.java: ## @@ -57,20 +83,260 @@ public ListingBasedRollb

[GitHub] [hudi] xushiyan commented on a diff in pull request #5279: [HUDI-3843] Make flink profiles build with scala-2.11

2022-04-10 Thread GitBox
xushiyan commented on code in PR #5279: URL: https://github.com/apache/hudi/pull/5279#discussion_r846936763 ## scripts/release/deploy_staging_jars.sh: ## @@ -38,6 +38,7 @@ fi declare -a ALL_VERSION_OPTS=( "-Dscala-2.11 -Dspark2.4 -Dflink1.13" +"-Dscala-2.11 -Dspark2.4 -Dflin

[GitHub] [hudi] codope commented on a diff in pull request #5255: [HUDI-3798] Fixing ending of a transaction by different owner and removing some extraneous methods in trxn manager

2022-04-10 Thread GitBox
codope commented on code in PR #5255: URL: https://github.com/apache/hudi/pull/5255#discussion_r846941822 ## hudi-client/hudi-client-common/src/test/java/org/apache/hudi/client/transaction/TestTransactionManager.java: ## @@ -152,6 +165,32 @@ public void testMultiWriterTransactio

[GitHub] [hudi] codope merged pull request #5255: [HUDI-3798] Fixing ending of a transaction by different owner and removing some extraneous methods in trxn manager

2022-04-10 Thread GitBox
codope merged PR #5255: URL: https://github.com/apache/hudi/pull/5255 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.or

[hudi] branch master updated: [HUDI-3798] Fixing ending of a transaction by different owner and removing some extraneous methods in trxn manager (#5255)

2022-04-10 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 2245a9515f [HUDI-3798] Fixing ending of a transact

[GitHub] [hudi] XuQianJin-Stars commented on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

2022-04-10 Thread GitBox
XuQianJin-Stars commented on PR #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-1094551133 > left some comments I saw your comment, I will submit a pr to fix it after. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

  1   2   >