[hudi] annotated tag release-0.11.0-rc3 updated (25501c99e9 -> 84a39887ad)

2022-04-21 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a change to annotated tag release-0.11.0-rc3 in repository https://gitbox.apache.org/repos/asf/hudi.git *** WARNING: tag release-0.11.0-rc3 was modified! *** from 25501c99e9 (commit) to 84a39887ad (tag

[GitHub] [hudi] hudi-bot commented on pull request #5393: [MINOR] follow up HUDI-3921, address all comments

2022-04-21 Thread GitBox
hudi-bot commented on PR #5393: URL: https://github.com/apache/hudi/pull/5393#issuecomment-1106069267 ## CI report: * 82b433d183f1c034d69768b27c616fc862577907 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=822

[GitHub] [hudi] hudi-bot commented on pull request #5393: [MINOR] follow up HUDI-3921, address all comments

2022-04-21 Thread GitBox
hudi-bot commented on PR #5393: URL: https://github.com/apache/hudi/pull/5393#issuecomment-1106067323 ## CI report: * 82b433d183f1c034d69768b27c616fc862577907 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8227

[GitHub] [hudi] hudi-bot commented on pull request #5393: [MINOR] follow up HUDI-3921, address all comments

2022-04-21 Thread GitBox
hudi-bot commented on PR #5393: URL: https://github.com/apache/hudi/pull/5393#issuecomment-1106065323 ## CI report: * 82b433d183f1c034d69768b27c616fc862577907 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8227

[hudi] 01/05: [MINOR] Increase azure CI timeout to 120m (#5384)

2022-04-21 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch release-0.11.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit f8ce1c53eca05d319dccd073567e1ceaa921 Author: Raymond Xu <2701446+xushi...@users.noreply.github.com> AuthorDat

[hudi] 04/05: [HUDI-3936] Fix projection for a nested field as pre-combined key (#5379)

2022-04-21 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch release-0.11.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 5bd7a677b260f6252979a6f9d2245fc34e6e6b19 Author: Y Ethan Guo AuthorDate: Thu Apr 21 17:17:57 2022 -0700 [HU

[hudi] branch release-0.11.0 updated (6fccca6c04 -> 25501c99e9)

2022-04-21 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a change to branch release-0.11.0 in repository https://gitbox.apache.org/repos/asf/hudi.git from 6fccca6c04 [HUDI-3935] Adding config to fallback to enabled Partition Values extraction from Partition path (

[hudi] 02/05: [HUDI-3940] Fix retry count increment in lock manager (#5387)

2022-04-21 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch release-0.11.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 304c5bf20ee6c8c45cb3d03213b52d6ea85530c4 Author: Sagar Sumit AuthorDate: Fri Apr 22 02:22:05 2022 +0530 [HU

[hudi] 03/05: [HUDI-3921] Fixed schema evolution cannot work with HUDI-3855 (#5376)

2022-04-21 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch release-0.11.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 0fb2d5bae12e6d6e10ba929e47c361997829b708 Author: xiarixiaoyao AuthorDate: Fri Apr 22 06:27:54 2022 +0800 [H

[hudi] 05/05: [HUDI-3934] Fix `Spark32HoodieParquetFileFormat` not being compatible w/ Spark 3.2.0 (#5378)

2022-04-21 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch release-0.11.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 25501c99e9e353bb8cf3757404cc0cd1835e03b3 Author: Alexey Kudinkin AuthorDate: Thu Apr 21 18:00:38 2022 -0700

[GitHub] [hudi] xiarixiaoyao commented on a diff in pull request #5376: [HUDI-3921] Fixed schema evolution cannot work with HUDI-3855

2022-04-21 Thread GitBox
xiarixiaoyao commented on code in PR #5376: URL: https://github.com/apache/hudi/pull/5376#discussion_r855810475 ## hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java: ## @@ -405,6 +407,14 @@ public static GenericRecord rewriteRecordWithMetadata(GenericRecord ge

[GitHub] [hudi] hudi-bot commented on pull request #5393: [MINOR] follow up HUDI-3921, address all comments

2022-04-21 Thread GitBox
hudi-bot commented on PR #5393: URL: https://github.com/apache/hudi/pull/5393#issuecomment-1106063330 ## CI report: * 82b433d183f1c034d69768b27c616fc862577907 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] xiarixiaoyao commented on a diff in pull request #5376: [HUDI-3921] Fixed schema evolution cannot work with HUDI-3855

2022-04-21 Thread GitBox
xiarixiaoyao commented on code in PR #5376: URL: https://github.com/apache/hudi/pull/5376#discussion_r855810204 ## hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java: ## @@ -719,14 +729,28 @@ public static Object getRecordColumnValues(HoodieRecord (colNameFromN

[GitHub] [hudi] xiarixiaoyao commented on a diff in pull request #5376: [HUDI-3921] Fixed schema evolution cannot work with HUDI-3855

2022-04-21 Thread GitBox
xiarixiaoyao commented on code in PR #5376: URL: https://github.com/apache/hudi/pull/5376#discussion_r855810004 ## hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java: ## @@ -765,27 +802,41 @@ private static Object rewriteRecordWithNewSchema(Object oldRecord, Sc

[GitHub] [hudi] xiarixiaoyao commented on a diff in pull request #5376: [HUDI-3921] Fixed schema evolution cannot work with HUDI-3855

2022-04-21 Thread GitBox
xiarixiaoyao commented on code in PR #5376: URL: https://github.com/apache/hudi/pull/5376#discussion_r855809877 ## hudi-common/src/main/java/org/apache/hudi/common/table/log/AbstractHoodieLogRecordReader.java: ## @@ -379,7 +380,7 @@ private void processDataBlock(HoodieDataBlock

[GitHub] [hudi] xiarixiaoyao commented on a diff in pull request #5376: [HUDI-3921] Fixed schema evolution cannot work with HUDI-3855

2022-04-21 Thread GitBox
xiarixiaoyao commented on code in PR #5376: URL: https://github.com/apache/hudi/pull/5376#discussion_r855809728 ## hudi-common/src/test/java/org/apache/hudi/internal/schema/utils/TestAvroSchemaEvolutionUtils.java: ## @@ -284,7 +284,7 @@ public void testReWriteRecordWithTypeChang

[GitHub] [hudi] xiarixiaoyao commented on a diff in pull request #5376: [HUDI-3921] Fixed schema evolution cannot work with HUDI-3855

2022-04-21 Thread GitBox
xiarixiaoyao commented on code in PR #5376: URL: https://github.com/apache/hudi/pull/5376#discussion_r855809616 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestSpark3DDL.scala: ## @@ -445,28 +445,19 @@ class TestSpark3DDL extends TestHoodieSqlBas

[GitHub] [hudi] xiarixiaoyao commented on pull request #5376: [HUDI-3921] Fixed schema evolution cannot work with HUDI-3855

2022-04-21 Thread GitBox
xiarixiaoyao commented on PR #5376: URL: https://github.com/apache/hudi/pull/5376#issuecomment-1106061517 @alexeykudinkin @nsivabalan fixed all comments on https://github.com/apache/hudi/pull/5393/files -- This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [hudi] xiarixiaoyao opened a new pull request, #5393: [MINOR] follow up HUDI-3921, address all comments

2022-04-21 Thread GitBox
xiarixiaoyao opened a new pull request, #5393: URL: https://github.com/apache/hudi/pull/5393 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the pu

[GitHub] [hudi] hudi-bot commented on pull request #5392: [HUDI-3942] [RFC-50] Improve Timeline Server

2022-04-21 Thread GitBox
hudi-bot commented on PR #5392: URL: https://github.com/apache/hudi/pull/5392#issuecomment-1106043259 ## CI report: * 0b65ac83fdefc35f8f627c2cce6887a16487f528 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8226

[GitHub] [hudi] hudi-bot commented on pull request #5391: [HUDI-3945] After the async compaction operation is complete, the tas…

2022-04-21 Thread GitBox
hudi-bot commented on PR #5391: URL: https://github.com/apache/hudi/pull/5391#issuecomment-1106025404 ## CI report: * 71792295fe72e9a2c988bb4a2989e24c44cd85d0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8225

[GitHub] [hudi] hudi-bot commented on pull request #5390: [HUDI-3944] Supports CTAS compatibility with string, char, and varchar.

2022-04-21 Thread GitBox
hudi-bot commented on PR #5390: URL: https://github.com/apache/hudi/pull/5390#issuecomment-1106004999 ## CI report: * a3b8d3fd07963241670317ddd199d3f42cdd1b6f Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8224

[GitHub] [hudi] BalaMahesh commented on issue #4230: [SUPPORT] org.apache.hudi.exception.HoodieRemoteException: Failed to create marker file

2022-04-21 Thread GitBox
BalaMahesh commented on issue #4230: URL: https://github.com/apache/hudi/issues/4230#issuecomment-1106003184 @yihua this seems not yet merged in the master, any idea when this will be merged into master, so that we can pull and try it out. -- This is an automated message from the Apache G

[GitHub] [hudi] hudi-bot commented on pull request #5392: [HUDI-3942] [RFC-50] Improve Timeline Server

2022-04-21 Thread GitBox
hudi-bot commented on PR #5392: URL: https://github.com/apache/hudi/pull/5392#issuecomment-1105989142 ## CI report: * 0b65ac83fdefc35f8f627c2cce6887a16487f528 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8226

[GitHub] [hudi] hudi-bot commented on pull request #5392: [HUDI-3942] [RFC-50] Improve Timeline Server

2022-04-21 Thread GitBox
hudi-bot commented on PR #5392: URL: https://github.com/apache/hudi/pull/5392#issuecomment-1105987440 ## CI report: * 0b65ac83fdefc35f8f627c2cce6887a16487f528 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[jira] [Updated] (HUDI-3942) RFC-50 Improve timeline server

2022-04-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3942: - Labels: pull-request-available (was: ) > RFC-50 Improve timeline server > ---

[GitHub] [hudi] yuzhaojing opened a new pull request, #5392: [HUDI-3942] [RFC-50] Improve Timeline Server

2022-04-21 Thread GitBox
yuzhaojing opened a new pull request, #5392: URL: https://github.com/apache/hudi/pull/5392 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purp

[GitHub] [hudi] yesemsanthoshkumar commented on issue #1985: [SUPPORT]Error while running deltastreamer on top of backfilled data using Hudi

2022-04-21 Thread GitBox
yesemsanthoshkumar commented on issue #1985: URL: https://github.com/apache/hudi/issues/1985#issuecomment-1105979967 @bvaradar In my usecase, the initial data is being loaded using spark.write() to hudi. Further data is flowing through kafka which I'd like to ingest on the same table. Steps

[GitHub] [hudi] hudi-bot commented on pull request #5391: [HUDI-3945] After the async compaction operation is complete, the tas…

2022-04-21 Thread GitBox
hudi-bot commented on PR #5391: URL: https://github.com/apache/hudi/pull/5391#issuecomment-1105973489 ## CI report: * 71792295fe72e9a2c988bb4a2989e24c44cd85d0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8225

[GitHub] [hudi] hudi-bot commented on pull request #5391: [HUDI-3945] After the async compaction operation is complete, the tas…

2022-04-21 Thread GitBox
hudi-bot commented on PR #5391: URL: https://github.com/apache/hudi/pull/5391#issuecomment-1105972203 ## CI report: * 71792295fe72e9a2c988bb4a2989e24c44cd85d0 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[jira] [Updated] (HUDI-3945) After the async compaction operation is complete, the task should exit.

2022-04-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3945: - Labels: pull-request-available (was: ) > After the async compaction operation is complete, the ta

[GitHub] [hudi] watermelon12138 opened a new pull request, #5391: [HUDI-3945] After the async compaction operation is complete, the tas…

2022-04-21 Thread GitBox
watermelon12138 opened a new pull request, #5391: URL: https://github.com/apache/hudi/pull/5391 …k should exit. ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.

[jira] [Assigned] (HUDI-3943) Some description fixes for 0.10.1 docs

2022-04-21 Thread Chuang Lee (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chuang Lee reassigned HUDI-3943: Assignee: Chuang Lee > Some description fixes for 0.10.1 docs > ---

[jira] [Created] (HUDI-3945) After the async compaction operation is complete, the task should exit.

2022-04-21 Thread YangXuan (Jira)
YangXuan created HUDI-3945: -- Summary: After the async compaction operation is complete, the task should exit. Key: HUDI-3945 URL: https://issues.apache.org/jira/browse/HUDI-3945 Project: Apache Hudi

[jira] [Closed] (HUDI-3693) error will be occured when writing data into hudi using flink job more than once in same flink session

2022-04-21 Thread chacha.tang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chacha.tang closed HUDI-3693. - Resolution: Invalid > error will be occured when writing data into hudi using flink job more than > once

[jira] [Updated] (HUDI-3693) error will be occured when writing data into hudi using flink job more than once in same flink session

2022-04-21 Thread chacha.tang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chacha.tang updated HUDI-3693: -- Status: Open (was: In Progress) > error will be occured when writing data into hudi using flink job mor

[GitHub] [hudi] hudi-bot commented on pull request #5390: [HUDI-3944] Supports CTAS compatibility with string, char, and varchar.

2022-04-21 Thread GitBox
hudi-bot commented on PR #5390: URL: https://github.com/apache/hudi/pull/5390#issuecomment-1105968535 ## CI report: * a3b8d3fd07963241670317ddd199d3f42cdd1b6f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8224

[GitHub] [hudi] hudi-bot commented on pull request #5390: [HUDI-3944] Supports CTAS compatibility with string, char, and varchar.

2022-04-21 Thread GitBox
hudi-bot commented on PR #5390: URL: https://github.com/apache/hudi/pull/5390#issuecomment-1105967043 ## CI report: * a3b8d3fd07963241670317ddd199d3f42cdd1b6f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[jira] [Updated] (HUDI-3944) Supports CTAS compatibility with string, char, and varchar.

2022-04-21 Thread ASF GitHub Bot (Jira)

[GitHub] [hudi] watermelon12138 opened a new pull request, #5390: [HUDI-3944] Supports CTAS compatibility with string, char, and varchar.

2022-04-21 Thread GitBox
watermelon12138 opened a new pull request, #5390: URL: https://github.com/apache/hudi/pull/5390 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

[jira] [Updated] (HUDI-3944) Supports CTAS compatibility with string, char, and varchar.

2022-04-21 Thread YangXuan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YangXuan updated HUDI-3944: --- Description: 1、create a table containing fields of type char and type varchar use hive. create table hivetb_c

[jira] [Created] (HUDI-3944) Supports CTAS compatibility with string, char, and varchar.

2022-04-21 Thread YangXuan (Jira)
YangXuan created HUDI-3944: -- Summary: Supports CTAS compatibility with string, char, and varchar. Key: HUDI-3944 URL: https://issues.apache.org/jira/browse/HUDI-3944 Project: Apache Hudi Issue Type

[jira] [Created] (HUDI-3943) Some description fixes for 0.10.1 docs

2022-04-21 Thread Chuang Lee (Jira)
Chuang Lee created HUDI-3943: Summary: Some description fixes for 0.10.1 docs Key: HUDI-3943 URL: https://issues.apache.org/jira/browse/HUDI-3943 Project: Apache Hudi Issue Type: Improvement

[jira] [Closed] (HUDI-2948) Hudi Clustering Performance

2022-04-21 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin closed HUDI-2948. - Resolution: Fixed > Hudi Clustering Performance > --- > >

[jira] [Created] (HUDI-3942) RFC-50 Improve timeline server

2022-04-21 Thread Zhaojing Yu (Jira)
Zhaojing Yu created HUDI-3942: - Summary: RFC-50 Improve timeline server Key: HUDI-3942 URL: https://issues.apache.org/jira/browse/HUDI-3942 Project: Apache Hudi Issue Type: New Feature

[jira] [Closed] (HUDI-3896) Support Spark optimizations for `HadoopFsRelation`

2022-04-21 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin closed HUDI-3896. - Resolution: Duplicate > Support Spark optimizations for `HadoopFsRelation` > -

[jira] [Closed] (HUDI-3936) When precombine field is nested, MOR query breaks

2022-04-21 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin closed HUDI-3936. - Resolution: Fixed > When precombine field is nested, MOR query breaks > --

[jira] [Closed] (HUDI-3934) Spark datasource read for spark3.2.0 fails w/ latest master

2022-04-21 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin closed HUDI-3934. - Resolution: Fixed > Spark datasource read for spark3.2.0 fails w/ latest master >

[jira] [Updated] (HUDI-3934) Spark datasource read for spark3.2.0 fails w/ latest master

2022-04-21 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3934: -- Sprint: Hudi-Sprint-Apr-19 > Spark datasource read for spark3.2.0 fails w/ latest master > -

[jira] [Closed] (HUDI-3935) Add config to fallback to extracting Partition Values from Partition Path

2022-04-21 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin closed HUDI-3935. - Resolution: Fixed > Add config to fallback to extracting Partition Values from Partition Path > --

[jira] [Resolved] (HUDI-3848) Restore fails when files pertaining to a commit has been cleaned up

2022-04-21 Thread Forward Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Forward Xu resolved HUDI-3848. -- > Restore fails when files pertaining to a commit has been cleaned up >

[GitHub] [hudi] xiarixiaoyao commented on pull request #5376: [HUDI-3921] Fixed schema evolution cannot work with HUDI-3855

2022-04-21 Thread GitBox
xiarixiaoyao commented on PR #5376: URL: https://github.com/apache/hudi/pull/5376#issuecomment-1105914395 @nsivabalan @alexeykudinkin thanks for your review, let me put another pr to optimize the code -- This is an automated message from the Apache Git Service. To respond to the message,

[hudi] branch release-0.11-0-apr21 updated: [HUDI-3934] Fix `Spark32HoodieParquetFileFormat` not being compatible w/ Spark 3.2.0 (#5378)

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.11-0-apr21 in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/release-0.11-0-apr21 by this push: new f21899fe25 [HUDI-3

[hudi] branch master updated (c4bc2deea0 -> c05a4e7b6f)

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from c4bc2deea0 [HUDI-3936] Fix projection for a nested field as pre-combined key (#5379) add c05a4e7b6f [HUDI-3934]

[GitHub] [hudi] nsivabalan merged pull request #5378: [HUDI-3934] Fix `Spark32HoodieParquetFileFormat` not being compatible w/ Spark 3.2.0

2022-04-21 Thread GitBox
nsivabalan merged PR #5378: URL: https://github.com/apache/hudi/pull/5378 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apach

[GitHub] [hudi] hudi-bot commented on pull request #5378: [HUDI-3934] Fix `Spark32HoodieParquetFileFormat` not being compatible w/ Spark 3.2.0

2022-04-21 Thread GitBox
hudi-bot commented on PR #5378: URL: https://github.com/apache/hudi/pull/5378#issuecomment-1105900280 ## CI report: * c03a0c5c5c3b0072888b6ca80ca6d6f84e508a99 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8221

[GitHub] [hudi] hudi-bot commented on pull request #5378: [HUDI-3934] Fix `Spark32HoodieParquetFileFormat` not being compatible w/ Spark 3.2.0

2022-04-21 Thread GitBox
hudi-bot commented on PR #5378: URL: https://github.com/apache/hudi/pull/5378#issuecomment-1105896181 ## CI report: * c03a0c5c5c3b0072888b6ca80ca6d6f84e508a99 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8221

[GitHub] [hudi] hudi-bot commented on pull request #5378: [HUDI-3934] Fix `Spark32HoodieParquetFileFormat` not being compatible w/ Spark 3.2.0

2022-04-21 Thread GitBox
hudi-bot commented on PR #5378: URL: https://github.com/apache/hudi/pull/5378#issuecomment-1105894082 ## CI report: * c03a0c5c5c3b0072888b6ca80ca6d6f84e508a99 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8220

[GitHub] [hudi] hudi-bot commented on pull request #5378: [HUDI-3934] Fix `Spark32HoodieParquetFileFormat` not being compatible w/ Spark 3.2.0

2022-04-21 Thread GitBox
hudi-bot commented on PR #5378: URL: https://github.com/apache/hudi/pull/5378#issuecomment-1105891991 ## CI report: * c03a0c5c5c3b0072888b6ca80ca6d6f84e508a99 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[hudi] 07/15: Fallback to pick ctors positionally as opposed to looking them up

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.11-0-apr21-5378-patched in repository https://gitbox.apache.org/repos/asf/hudi.git commit 286628908191740765319043664697df8e1877ec Author: Alexey Kudinkin AuthorDate: Wed Apr 20 15:3

[hudi] 03/15: Extracted `int96RebaseMode`, `datetimeRebaseMode` into `Spark32DataSourceUtils`

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.11-0-apr21-5378-patched in repository https://gitbox.apache.org/repos/asf/hudi.git commit 33ab74c354dd8c7b63b22ff1cc54287b9a3a80b6 Author: Alexey Kudinkin AuthorDate: Wed Apr 20 14:4

[hudi] 10/15: use tail to get VectorizedParquetRecordReader ctor

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.11-0-apr21-5378-patched in repository https://gitbox.apache.org/repos/asf/hudi.git commit 5d0556745ed555bebd472250a0de3b9cbcbe7f15 Author: Raymond Xu <2701446+xushi...@users.noreply.g

[hudi] 09/15: Revert "Cleaning up `ReflectionUtils`"

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.11-0-apr21-5378-patched in repository https://gitbox.apache.org/repos/asf/hudi.git commit b77f026b5498d55cc26d2ab7ed9059801d0d37b2 Author: Alexey Kudinkin AuthorDate: Wed Apr 20 22:2

[hudi] 08/15: `Spark312HoodieParquetFileFormat` > `Spark31HoodieParquetFileFormat`

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.11-0-apr21-5378-patched in repository https://gitbox.apache.org/repos/asf/hudi.git commit 02985a45c8f29db626f9d41e67c40f9662d66069 Author: Alexey Kudinkin AuthorDate: Wed Apr 20 15:3

[hudi] 11/15: fix tail misuse

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.11-0-apr21-5378-patched in repository https://gitbox.apache.org/repos/asf/hudi.git commit 1dfab33b7a53b0e02690dd45320d2bb86980003a Author: Raymond Xu <2701446+xushi...@users.noreply.g

[hudi] 14/15: Hardening the code

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.11-0-apr21-5378-patched in repository https://gitbox.apache.org/repos/asf/hudi.git commit 12b9b783b9c74519044200592d897bf90efcf3ad Author: Alexey Kudinkin AuthorDate: Thu Apr 21 13:0

[hudi] 13/15: Fixing assumption that HoodieParquetFileFormat could only be used w/ Hudi tables

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.11-0-apr21-5378-patched in repository https://gitbox.apache.org/repos/asf/hudi.git commit 4be545a28501b0a56064f19e2af768b4f3a3b925 Author: Alexey Kudinkin AuthorDate: Thu Apr 21 13:0

[hudi] 05/15: Fixed instantiation of the components t/h reflection

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.11-0-apr21-5378-patched in repository https://gitbox.apache.org/repos/asf/hudi.git commit bc58383dd1232e8957427b89aaf3f5a83fe17b62 Author: Alexey Kudinkin AuthorDate: Wed Apr 20 15:2

[hudi] 01/15: Re-use `ReflectionUtils`

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.11-0-apr21-5378-patched in repository https://gitbox.apache.org/repos/asf/hudi.git commit 689cf2c588da4c722d4d538ffb34600b638834ea Author: Alexey Kudinkin AuthorDate: Wed Apr 20 13:1

[hudi] 04/15: Added new reflection utisl to instantiate already loaded class

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.11-0-apr21-5378-patched in repository https://gitbox.apache.org/repos/asf/hudi.git commit 1cbc580f3a3db16142a42f0d8a8dbcf9bedbc063 Author: Alexey Kudinkin AuthorDate: Wed Apr 20 15:1

[hudi] 06/15: Fixing NPEs

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.11-0-apr21-5378-patched in repository https://gitbox.apache.org/repos/asf/hudi.git commit f697fb1e8ebe14c1c906099049efe406cbb8dc5c Author: Alexey Kudinkin AuthorDate: Wed Apr 20 15:2

[hudi] branch release-0.11-0-apr21-5378-patched created (now ae095b9a56)

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch release-0.11-0-apr21-5378-patched in repository https://gitbox.apache.org/repos/asf/hudi.git at ae095b9a56 Replicating to Spark 3.1 This branch includes the following new commits:

[hudi] 15/15: Replicating to Spark 3.1

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.11-0-apr21-5378-patched in repository https://gitbox.apache.org/repos/asf/hudi.git commit ae095b9a56ef2a25b8bb75406a51354b9f4ee13f Author: Alexey Kudinkin AuthorDate: Thu Apr 21 13:0

[hudi] 02/15: Handle incompatibilities b/w Spark 3.2.0 and 3.2.1 in `Spark32HoodieParquetFileFormat`

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.11-0-apr21-5378-patched in repository https://gitbox.apache.org/repos/asf/hudi.git commit bdcbbe710fdc26cbd65613353b60a91f75716c90 Author: Alexey Kudinkin AuthorDate: Wed Apr 20 14:2

[hudi] 12/15: fix undeterministic ctor order

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.11-0-apr21-5378-patched in repository https://gitbox.apache.org/repos/asf/hudi.git commit f0beb5311b65267eca6d6b96e5182f28a83dea5d Author: Raymond Xu <2701446+xushi...@users.noreply.g

[hudi] 02/04: [HUDI-3940] Fix retry count increment in lock manager (#5387)

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.11-0-apr21 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 29c1dca5fd739004b2c24ac81e63ffa99a81014d Author: Sagar Sumit AuthorDate: Fri Apr 22 02:22:05 2022 +0530

[hudi] 03/04: [HUDI-3921] Fixed schema evolution cannot work with HUDI-3855 (#5376)

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.11-0-apr21 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 544d9f4b5d47c826fd5d57f874bb4341e8e93637 Author: xiarixiaoyao AuthorDate: Fri Apr 22 06:27:54 2022 +0800

[hudi] branch release-0.11-0-apr21 created (now 6fe5c27a0c)

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch release-0.11-0-apr21 in repository https://gitbox.apache.org/repos/asf/hudi.git at 6fe5c27a0c [HUDI-3936] Fix projection for a nested field as pre-combined key (#5379) This branch incl

[hudi] 04/04: [HUDI-3936] Fix projection for a nested field as pre-combined key (#5379)

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.11-0-apr21 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 6fe5c27a0c080ebb79eec6544c0dc5cacc028919 Author: Y Ethan Guo AuthorDate: Thu Apr 21 17:17:57 2022 -0700

[hudi] 01/04: [MINOR] Increase azure CI timeout to 120m (#5384)

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch release-0.11-0-apr21 in repository https://gitbox.apache.org/repos/asf/hudi.git commit edf2d0e788ba48450f48027cd0c3761ae45eb3ef Author: Raymond Xu <2701446+xushi...@users.noreply.github.com> Au

[hudi] branch master updated (037f89ee7c -> c4bc2deea0)

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 037f89ee7c [HUDI-3921] Fixed schema evolution cannot work with HUDI-3855 (#5376) add c4bc2deea0 [HUDI-3936] Fix

[GitHub] [hudi] nsivabalan merged pull request #5379: [HUDI-3936] Fix projection for a nested field as pre-combined key

2022-04-21 Thread GitBox
nsivabalan merged PR #5379: URL: https://github.com/apache/hudi/pull/5379 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apach

[GitHub] [hudi] hudi-bot commented on pull request #5379: [HUDI-3936] Fix projection for a nested field as pre-combined key

2022-04-21 Thread GitBox
hudi-bot commented on PR #5379: URL: https://github.com/apache/hudi/pull/5379#issuecomment-1105867957 ## CI report: * 5aae2f9d2e69727ca24605ecea2e3c74d04d3f00 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=821

[GitHub] [hudi] hudi-bot commented on pull request #5378: [HUDI-3934] Fix `Spark32HoodieParquetFileFormat` not being compatible w/ Spark 3.2.0

2022-04-21 Thread GitBox
hudi-bot commented on PR #5378: URL: https://github.com/apache/hudi/pull/5378#issuecomment-1105829316 ## CI report: * 3017818e80a7c4969d57c279bea2ea91b9d9c9bf Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=820

[GitHub] [hudi] hudi-bot commented on pull request #5378: [HUDI-3934] Fix `Spark32HoodieParquetFileFormat` not being compatible w/ Spark 3.2.0

2022-04-21 Thread GitBox
hudi-bot commented on PR #5378: URL: https://github.com/apache/hudi/pull/5378#issuecomment-1105827596 ## CI report: * c03a0c5c5c3b0072888b6ca80ca6d6f84e508a99 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=821

[GitHub] [hudi] alexeykudinkin commented on pull request #5378: [HUDI-3934] Fix `Spark32HoodieParquetFileFormat` not being compatible w/ Spark 3.2.0

2022-04-21 Thread GitBox
alexeykudinkin commented on PR #5378: URL: https://github.com/apache/hudi/pull/5378#issuecomment-1105826504 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

[hudi] branch master updated (de5fa1fe03 -> 037f89ee7c)

2022-04-21 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from de5fa1fe03 [HUDI-3940] Fix retry count increment in lock manager (#5387) add 037f89ee7c [HUDI-3921] Fixed schem

[GitHub] [hudi] nsivabalan merged pull request #5376: [HUDI-3921] Fixed schema evolution cannot work with HUDI-3855

2022-04-21 Thread GitBox
nsivabalan merged PR #5376: URL: https://github.com/apache/hudi/pull/5376 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apach

[GitHub] [hudi] nsivabalan commented on pull request #5376: [HUDI-3921] Fixed schema evolution cannot work with HUDI-3855

2022-04-21 Thread GitBox
nsivabalan commented on PR #5376: URL: https://github.com/apache/hudi/pull/5376#issuecomment-1105821432 @xiarixiaoyao : please address the feedback in a follow up PR. I am going ahead and landing this. -- This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #5376: [HUDI-3921] Fixed schema evolution cannot work with HUDI-3855

2022-04-21 Thread GitBox
alexeykudinkin commented on code in PR #5376: URL: https://github.com/apache/hudi/pull/5376#discussion_r855624045 ## hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java: ## @@ -741,10 +765,23 @@ private static Object rewriteRecordWithNewSchema(Object oldRecord,

[GitHub] [hudi] hudi-bot commented on pull request #5379: [HUDI-3936] Fix projection for a nested field as pre-combined key

2022-04-21 Thread GitBox
hudi-bot commented on PR #5379: URL: https://github.com/apache/hudi/pull/5379#issuecomment-1105797898 ## CI report: * 5aae2f9d2e69727ca24605ecea2e3c74d04d3f00 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=821

[GitHub] [hudi] alexeykudinkin commented on pull request #5379: [HUDI-3936] Fix projection for a nested field as pre-combined key

2022-04-21 Thread GitBox
alexeykudinkin commented on PR #5379: URL: https://github.com/apache/hudi/pull/5379#issuecomment-1105797145 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

[GitHub] [hudi] nsivabalan commented on a diff in pull request #5376: [HUDI-3921] Fixed schema evolution cannot work with HUDI-3855

2022-04-21 Thread GitBox
nsivabalan commented on code in PR #5376: URL: https://github.com/apache/hudi/pull/5376#discussion_r855614094 ## hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java: ## @@ -405,6 +407,18 @@ public static GenericRecord rewriteRecordWithMetadata(GenericRecord gene

[GitHub] [hudi] nsivabalan commented on a diff in pull request #5376: [HUDI-3921] Fixed schema evolution cannot work with HUDI-3855

2022-04-21 Thread GitBox
nsivabalan commented on code in PR #5376: URL: https://github.com/apache/hudi/pull/5376#discussion_r855613640 ## hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java: ## @@ -741,10 +769,23 @@ private static Object rewriteRecordWithNewSchema(Object oldRecord, Sche

[GitHub] [hudi] hudi-bot commented on pull request #5376: [HUDI-3921] Fixed schema evolution cannot work with HUDI-3855

2022-04-21 Thread GitBox
hudi-bot commented on PR #5376: URL: https://github.com/apache/hudi/pull/5376#issuecomment-1105775359 ## CI report: * cbdc49842258501c1888e5cc247c70fadfde20b7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8203

[GitHub] [hudi] vinothchandar commented on a diff in pull request #5113: [HUDI-3625] [RFC-48] Optimized storage layout for Cloud Object Stores

2022-04-21 Thread GitBox
vinothchandar commented on code in PR #5113: URL: https://github.com/apache/hudi/pull/5113#discussion_r855589377 ## rfc/rfc-48/rfc-48.md: ## @@ -0,0 +1,171 @@ + + +# RFC-[48]: Optimized storage layout for Cloud Object Stores + +## Proposers +- @umehrot2 + +## Approvers +- @vinot

[GitHub] [hudi] hudi-bot commented on pull request #5378: [HUDI-3934] Fix `Spark32HoodieParquetFileFormat` not being compatible w/ Spark 3.2.0

2022-04-21 Thread GitBox
hudi-bot commented on PR #5378: URL: https://github.com/apache/hudi/pull/5378#issuecomment-1105772380 ## CI report: * 3017818e80a7c4969d57c279bea2ea91b9d9c9bf Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=820

[GitHub] [hudi] hudi-bot commented on pull request #5376: [HUDI-3921] Fixed schema evolution cannot work with HUDI-3855

2022-04-21 Thread GitBox
hudi-bot commented on PR #5376: URL: https://github.com/apache/hudi/pull/5376#issuecomment-1105772333 ## CI report: * cbdc49842258501c1888e5cc247c70fadfde20b7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8203

[GitHub] [hudi] hudi-bot commented on pull request #5378: [HUDI-3934] Fix `Spark32HoodieParquetFileFormat` not being compatible w/ Spark 3.2.0

2022-04-21 Thread GitBox
hudi-bot commented on PR #5378: URL: https://github.com/apache/hudi/pull/5378#issuecomment-1105769610 ## CI report: * 6409d526040c06399ad8085f760a8d9537eb26c1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8202

[GitHub] [hudi] hudi-bot commented on pull request #5379: [HUDI-3936] Fix projection for a nested field as pre-combined key

2022-04-21 Thread GitBox
hudi-bot commented on PR #5379: URL: https://github.com/apache/hudi/pull/5379#issuecomment-1105760787 ## CI report: * Unknown: [CANCELED](TBD) * 5aae2f9d2e69727ca24605ecea2e3c74d04d3f00 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039

  1   2   3   >