[jira] [Created] (HUDI-2951) Disable remote view storage config for flink

2021-12-06 Thread Danny Chen (Jira)
Danny Chen created HUDI-2951: Summary: Disable remote view storage config for flink Key: HUDI-2951 URL: https://issues.apache.org/jira/browse/HUDI-2951 Project: Apache Hudi Issue Type: Bug

[GitHub] [hudi] hudi-bot commented on pull request #4193: [HUDI-2915] Fix field not found error for sparksql

2021-12-06 Thread GitBox
hudi-bot commented on pull request #4193: URL: https://github.com/apache/hudi/pull/4193#issuecomment-987630193 ## CI report: * 78c68edc8debb60e13b15883597dedcfa37f67ba UNKNOWN * 78527e7981827a05258ac476363c77ac85ff6e98 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/

[GitHub] [hudi] hudi-bot removed a comment on pull request #4193: [HUDI-2915] Fix field not found error for sparksql

2021-12-06 Thread GitBox
hudi-bot removed a comment on pull request #4193: URL: https://github.com/apache/hudi/pull/4193#issuecomment-987611616 ## CI report: * 78c68edc8debb60e13b15883597dedcfa37f67ba UNKNOWN * 78527e7981827a05258ac476363c77ac85ff6e98 Azure: [FAILURE](https://dev.azure.com/apache-hudi

[GitHub] [hudi] h7kanna edited a comment on issue #2688: [SUPPORT] Sync to Hive using Metastore

2021-12-06 Thread GitBox
h7kanna edited a comment on issue #2688: URL: https://github.com/apache/hudi/issues/2688#issuecomment-987628251 I can confirm this works with mode='hms' and hive_jdbc=false ```shell spark-shell \ --conf "spark.serializer=org.apache.spark.serializer.KryoSerializer" \ --jars /

[GitHub] [hudi] h7kanna commented on issue #2688: [SUPPORT] Sync to Hive using Metastore

2021-12-06 Thread GitBox
h7kanna commented on issue #2688: URL: https://github.com/apache/hudi/issues/2688#issuecomment-987628251 I can confirm this works with mode='hms' and hive_jdbc=false spark-shell \ > --conf "spark.serializer=org.apache.spark.serializer.KryoSerializer" \ > --jars /home/hadoop/hud

[GitHub] [hudi] hudi-bot commented on pull request #4236: [HUDI-2936] Add data count checks in async clustering tests

2021-12-06 Thread GitBox
hudi-bot commented on pull request #4236: URL: https://github.com/apache/hudi/pull/4236#issuecomment-987617704 ## CI report: * e4908379cb7faee6bdc554b0937b9a4557797eea Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4236: [HUDI-2936] Add data count checks in async clustering tests

2021-12-06 Thread GitBox
hudi-bot removed a comment on pull request #4236: URL: https://github.com/apache/hudi/pull/4236#issuecomment-987593161 ## CI report: * e4908379cb7faee6bdc554b0937b9a4557797eea Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[jira] [Assigned] (HUDI-1846) Add hudi-python module and add python API into it

2021-12-06 Thread Yuepeng Pan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuepeng Pan reassigned HUDI-1846: - Assignee: (was: Yuepeng Pan) > Add hudi-python module and add python API into it > --

[GitHub] [hudi] stym06 commented on issue #3961: [SUPPORT] Hudi DeltaStreamer failing during Cleaning operation

2021-12-06 Thread GitBox
stym06 commented on issue #3961: URL: https://github.com/apache/hudi/issues/3961#issuecomment-987615551 Sure, Do you need the complete .hoodie folder with its contents? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] hudi-bot commented on pull request #4193: [HUDI-2915] Fix field not found error for sparksql

2021-12-06 Thread GitBox
hudi-bot commented on pull request #4193: URL: https://github.com/apache/hudi/pull/4193#issuecomment-987611616 ## CI report: * 78c68edc8debb60e13b15883597dedcfa37f67ba UNKNOWN * 78527e7981827a05258ac476363c77ac85ff6e98 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/

[GitHub] [hudi] hudi-bot removed a comment on pull request #4193: [HUDI-2915] Fix field not found error for sparksql

2021-12-06 Thread GitBox
hudi-bot removed a comment on pull request #4193: URL: https://github.com/apache/hudi/pull/4193#issuecomment-987545725 ## CI report: * 78c68edc8debb60e13b15883597dedcfa37f67ba UNKNOWN * 78527e7981827a05258ac476363c77ac85ff6e98 Azure: [FAILURE](https://dev.azure.com/apache-hudi

[jira] [Updated] (HUDI-2367) Handle deletes in S3 deltastreamer source

2021-12-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2367: -- Summary: Handle deletes in S3 deltastreamer source (was: Handle deletes in S3 deltastre

[GitHub] [hudi] nsivabalan commented on pull request #3946: [HUDI-2711] Fallback to fulltable scan for IncrementalRelation if underlying files have been cleared or moved by cleaner

2021-12-06 Thread GitBox
nsivabalan commented on pull request #3946: URL: https://github.com/apache/hudi/pull/3946#issuecomment-987598430 something to think about as a potential solution. applicable only if cleaner policy is set to num commits. if min and max archival commits is 5 and 10. and cleaner configs

[GitHub] [hudi] nsivabalan commented on a change in pull request #3946: [HUDI-2711] Fallback to fulltable scan for IncrementalRelation if underlying files have been cleared or moved by cleaner

2021-12-06 Thread GitBox
nsivabalan commented on a change in pull request #3946: URL: https://github.com/apache/hudi/pull/3946#discussion_r763655793 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/IncrementalRelation.scala ## @@ -155,27 +155,60 @@ class IncrementalRelatio

[GitHub] [hudi] nsivabalan commented on a change in pull request #3946: [HUDI-2711] Fallback to fulltable scan for IncrementalRelation if underlying files have been cleared or moved by cleaner

2021-12-06 Thread GitBox
nsivabalan commented on a change in pull request #3946: URL: https://github.com/apache/hudi/pull/3946#discussion_r763655793 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/IncrementalRelation.scala ## @@ -155,27 +155,60 @@ class IncrementalRelatio

[GitHub] [hudi] nsivabalan commented on a change in pull request #3946: [HUDI-2711] Fallback to fulltable scan for IncrementalRelation if underlying files have been cleared or moved by cleaner

2021-12-06 Thread GitBox
nsivabalan commented on a change in pull request #3946: URL: https://github.com/apache/hudi/pull/3946#discussion_r763652433 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/IncrementalRelation.scala ## @@ -155,27 +155,60 @@ class IncrementalRelatio

[GitHub] [hudi] xuzifu666 edited a comment on pull request #4224: [HUDI-2942] add error message log in HoodieCombineHiveInputFormat

2021-12-06 Thread GitBox
xuzifu666 edited a comment on pull request #4224: URL: https://github.com/apache/hudi/pull/4224#issuecomment-986624239 @leesf @vinothchandar have a review when have time please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] nsivabalan commented on issue #3603: [SUPPORT] delta streamer Failed to archive commits

2021-12-06 Thread GitBox
nsivabalan commented on issue #3603: URL: https://github.com/apache/hudi/issues/3603#issuecomment-987593906 Will close it out if we don't hear from you in a day or two. please do let us know if you are having the issue still. -- This is an automated message from the Apache Git Service.

[GitHub] [hudi] nsivabalan commented on issue #3961: [SUPPORT] Hudi DeltaStreamer failing during Cleaning operation

2021-12-06 Thread GitBox
nsivabalan commented on issue #3961: URL: https://github.com/apache/hudi/issues/3961#issuecomment-987593327 @stym06 : can you respond to my clarification above. would help debug this issue further. -- This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [hudi] hudi-bot removed a comment on pull request #4236: [HUDI-2936] Add data count checks in async clustering tests

2021-12-06 Thread GitBox
hudi-bot removed a comment on pull request #4236: URL: https://github.com/apache/hudi/pull/4236#issuecomment-987592119 ## CI report: * e4908379cb7faee6bdc554b0937b9a4557797eea UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4236: [HUDI-2936] Add data count checks in async clustering tests

2021-12-06 Thread GitBox
hudi-bot commented on pull request #4236: URL: https://github.com/apache/hudi/pull/4236#issuecomment-987593161 ## CI report: * e4908379cb7faee6bdc554b0937b9a4557797eea Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot commented on pull request #4236: [HUDI-2936] Add data count checks in async clustering tests

2021-12-06 Thread GitBox
hudi-bot commented on pull request #4236: URL: https://github.com/apache/hudi/pull/4236#issuecomment-987592119 ## CI report: * e4908379cb7faee6bdc554b0937b9a4557797eea UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[GitHub] [hudi] hudi-bot commented on pull request #3989: [HUDI-2589] RFC-37: Metadata table based bloom index

2021-12-06 Thread GitBox
hudi-bot commented on pull request #3989: URL: https://github.com/apache/hudi/pull/3989#issuecomment-987591966 ## CI report: * d9213ef5e8ed41c242e1f9b9da710aa5a27e40b3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #3989: [HUDI-2589] RFC-37: Metadata table based bloom index

2021-12-06 Thread GitBox
hudi-bot removed a comment on pull request #3989: URL: https://github.com/apache/hudi/pull/3989#issuecomment-987572963 ## CI report: * 68cba72835e1e89f9c1b8096f5ae36724c48a9c1 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[jira] [Comment Edited] (HUDI-2943) Deltastreamer fails to continue with pending clustering after restart in 0.10.0

2021-12-06 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17454385#comment-17454385 ] Harsha Teja Kanna edited comment on HUDI-2943 at 12/7/21, 5:53 AM: -

[jira] [Commented] (HUDI-2943) Deltastreamer fails to continue with pending clustering after restart in 0.10.0

2021-12-06 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17454385#comment-17454385 ] Harsha Teja Kanna commented on HUDI-2943: - Yes. I am running latest master. Actua

[GitHub] [hudi] codope opened a new pull request #4236: [HUDI-2936] Add data count checks in async clustering tests

2021-12-06 Thread GitBox
codope opened a new pull request #4236: URL: https://github.com/apache/hudi/pull/4236 ## What is the purpose of the pull request This PR adds data count checks for correctness in async clustering tests for deltastreamer. Such checks are already present for spark datasource in ```

[jira] [Updated] (HUDI-2936) Add data correctness tests for async clustering

2021-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2936: - Labels: pull-request-available (was: ) > Add data correctness tests for async clustering > --

[GitHub] [hudi] yanghua commented on issue #4229: [SUPPORT] Exception in thread "main" java.lang.IllegalArgumentException: Can't find primaryKey `uuid` in root

2021-12-06 Thread GitBox
yanghua commented on issue #4229: URL: https://github.com/apache/hudi/issues/4229#issuecomment-987591321 Hi @BenjMaq Maybe, you should define the `uuid` field actively. Yes, the default behavior has been changed compared to 0.9. -- This is an automated message from the Apache Git Service

[GitHub] [hudi] hudi-bot removed a comment on pull request #3989: [HUDI-2589] RFC-37: Metadata table based bloom index

2021-12-06 Thread GitBox
hudi-bot removed a comment on pull request #3989: URL: https://github.com/apache/hudi/pull/3989#issuecomment-987571867 ## CI report: * 68cba72835e1e89f9c1b8096f5ae36724c48a9c1 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #3989: [HUDI-2589] RFC-37: Metadata table based bloom index

2021-12-06 Thread GitBox
hudi-bot commented on pull request #3989: URL: https://github.com/apache/hudi/pull/3989#issuecomment-987572963 ## CI report: * 68cba72835e1e89f9c1b8096f5ae36724c48a9c1 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot commented on pull request #3989: [HUDI-2589] RFC-37: Metadata table based bloom index

2021-12-06 Thread GitBox
hudi-bot commented on pull request #3989: URL: https://github.com/apache/hudi/pull/3989#issuecomment-987571867 ## CI report: * 68cba72835e1e89f9c1b8096f5ae36724c48a9c1 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #3989: [HUDI-2589] RFC-37: Metadata table based bloom index

2021-12-06 Thread GitBox
hudi-bot removed a comment on pull request #3989: URL: https://github.com/apache/hudi/pull/3989#issuecomment-969377387 ## CI report: * 68cba72835e1e89f9c1b8096f5ae36724c48a9c1 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[jira] [Commented] (HUDI-2943) Deltastreamer fails to continue with pending clustering after restart in 0.10.0

2021-12-06 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17454375#comment-17454375 ] Sagar Sumit commented on HUDI-2943: --- [~h7kanna] Did you try with latest master? We lande

[jira] [Updated] (HUDI-2821) Docs for Hudi Metadata

2021-12-06 Thread Kyle Weller (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Weller updated HUDI-2821: -- Status: Patch Available (was: In Progress) > Docs for Hudi Metadata > -- > >

[jira] [Commented] (HUDI-2735) Fix archival of commits in Java client for Kafka Connect

2021-12-06 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17454374#comment-17454374 ] Raymond Xu commented on HUDI-2735: -- Reproducible in  org.apache.hudi.io.TestHoodieTimelin

[GitHub] [hudi] xuzifu666 commented on a change in pull request #4224: [HUDI-2942] add error message log in HoodieCombineHiveInputFormat

2021-12-06 Thread GitBox
xuzifu666 commented on a change in pull request #4224: URL: https://github.com/apache/hudi/pull/4224#discussion_r762973856 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/hive/HoodieCombineHiveInputFormat.java ## @@ -183,8 +183,8 @@ protected HoodieCombineFil

[jira] [Updated] (HUDI-2827) Docs for schema provider

2021-12-06 Thread Kyle Weller (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Weller updated HUDI-2827: -- Status: Patch Available (was: In Progress) > Docs for schema provider > > >

[GitHub] [hudi] hudi-bot commented on pull request #4193: [HUDI-2915] Fix field not found error for sparksql

2021-12-06 Thread GitBox
hudi-bot commented on pull request #4193: URL: https://github.com/apache/hudi/pull/4193#issuecomment-987545725 ## CI report: * 78c68edc8debb60e13b15883597dedcfa37f67ba UNKNOWN * 78527e7981827a05258ac476363c77ac85ff6e98 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/

[GitHub] [hudi] hudi-bot removed a comment on pull request #4193: [HUDI-2915] Fix field not found error for sparksql

2021-12-06 Thread GitBox
hudi-bot removed a comment on pull request #4193: URL: https://github.com/apache/hudi/pull/4193#issuecomment-987522102 ## CI report: * 78c68edc8debb60e13b15883597dedcfa37f67ba UNKNOWN * 30d75e84f0b4b355d0aa8be00846609dfb292794 Azure: [SUCCESS](https://dev.azure.com/apache-hudi

[jira] [Commented] (HUDI-2941) Show _hoodie_operation in spark sql results

2021-12-06 Thread Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17454355#comment-17454355 ] Gary Li commented on HUDI-2941: --- [~lsy] just added you to the contributor list and assigned

[jira] [Assigned] (HUDI-2941) Show _hoodie_operation in spark sql results

2021-12-06 Thread Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Li reassigned HUDI-2941: - Assignee: dalongliu > Show _hoodie_operation in spark sql results > -

[jira] [Commented] (HUDI-2669) Upgrade jdk version to higher versions

2021-12-06 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17454353#comment-17454353 ] Alexey Kudinkin commented on HUDI-2669: --- Following settings will do the job {code:ja

[jira] [Updated] (HUDI-2886) Certify metadata table using large-scale cluster testing

2021-12-06 Thread Manoj Govindassamy (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HUDI-2886: - Status: Patch Available (was: In Progress) > Certify metadata table using large-scale clu

[GitHub] [hudi] hudi-bot commented on pull request #4193: [HUDI-2915] Fix field not found error for sparksql

2021-12-06 Thread GitBox
hudi-bot commented on pull request #4193: URL: https://github.com/apache/hudi/pull/4193#issuecomment-987522102 ## CI report: * 78c68edc8debb60e13b15883597dedcfa37f67ba UNKNOWN * 30d75e84f0b4b355d0aa8be00846609dfb292794 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/

[GitHub] [hudi] hudi-bot removed a comment on pull request #4193: [HUDI-2915] Fix field not found error for sparksql

2021-12-06 Thread GitBox
hudi-bot removed a comment on pull request #4193: URL: https://github.com/apache/hudi/pull/4193#issuecomment-987500232 ## CI report: * 78c68edc8debb60e13b15883597dedcfa37f67ba UNKNOWN * 30d75e84f0b4b355d0aa8be00846609dfb292794 Azure: [SUCCESS](https://dev.azure.com/apache-hudi

[GitHub] [hudi] XuQianJin-Stars closed pull request #4193: [HUDI-2915] Fix field not found error for sparksql

2021-12-06 Thread GitBox
XuQianJin-Stars closed pull request #4193: URL: https://github.com/apache/hudi/pull/4193 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-u

[jira] [Commented] (HUDI-2669) Upgrade jdk version to higher versions

2021-12-06 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17454333#comment-17454333 ] Alexey Kudinkin commented on HUDI-2669: --- Given how drastic difference in performance

[jira] [Updated] (HUDI-2669) Upgrade jdk version to higher versions

2021-12-06 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-2669: -- Description: We should upgrade to at least JDK11, or preferably current latest LTS JDK17    

[jira] [Commented] (HUDI-2669) Upgrade jdk version to higher versions

2021-12-06 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17454332#comment-17454332 ] Alexey Kudinkin commented on HUDI-2669: --- While perf-tuning [Layout Optimization (Clu

[jira] [Updated] (HUDI-2827) Docs for schema provider

2021-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2827: - Labels: pull-request-available (was: ) > Docs for schema provider > > >

[GitHub] [hudi] kywe665 opened a new pull request #4235: [HUDI-2827] - Docs for SchemaProviders and Sources

2021-12-06 Thread GitBox
kywe665 opened a new pull request #4235: URL: https://github.com/apache/hudi/pull/4235 ## What is the purpose of the pull request Added docs for SchemaProviders and Sources ## Brief change log - Added docs for SchemaProviders and Sources ## Verify this pull requ

[jira] [Updated] (HUDI-2827) Docs for schema provider

2021-12-06 Thread Kyle Weller (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Weller updated HUDI-2827: -- Fix Version/s: 0.10.0 > Docs for schema provider > > > Key: HUD

[jira] [Closed] (HUDI-2824) Improve DeltaStreamer Docs

2021-12-06 Thread Kyle Weller (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Weller closed HUDI-2824. - Resolution: Duplicate closing as dupe of schema providers and sources doc improvement > Improve DeltaStre

[GitHub] [hudi] hudi-bot commented on pull request #4193: [HUDI-2915] Fix field not found error for sparksql

2021-12-06 Thread GitBox
hudi-bot commented on pull request #4193: URL: https://github.com/apache/hudi/pull/4193#issuecomment-987500232 ## CI report: * 78c68edc8debb60e13b15883597dedcfa37f67ba UNKNOWN * 30d75e84f0b4b355d0aa8be00846609dfb292794 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/

[GitHub] [hudi] hudi-bot removed a comment on pull request #4193: [HUDI-2915] Fix field not found error for sparksql

2021-12-06 Thread GitBox
hudi-bot removed a comment on pull request #4193: URL: https://github.com/apache/hudi/pull/4193#issuecomment-985974206 ## CI report: * 78c68edc8debb60e13b15883597dedcfa37f67ba UNKNOWN * 30d75e84f0b4b355d0aa8be00846609dfb292794 Azure: [SUCCESS](https://dev.azure.com/apache-hudi

[GitHub] [hudi] hudi-bot commented on pull request #4234: [HUDI-2950][Stacked 4106] Addressing performance traps in Bulk Insert/Layout Optimization

2021-12-06 Thread GitBox
hudi-bot commented on pull request #4234: URL: https://github.com/apache/hudi/pull/4234#issuecomment-987499072 ## CI report: * 45b092771c86e8955133699742a1f9bb9cbc18ab Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4234: [HUDI-2950][Stacked 4106] Addressing performance traps in Bulk Insert/Layout Optimization

2021-12-06 Thread GitBox
hudi-bot removed a comment on pull request #4234: URL: https://github.com/apache/hudi/pull/4234#issuecomment-987497932 ## CI report: * 45b092771c86e8955133699742a1f9bb9cbc18ab Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4234: [HUDI-2950][Stacked 4106] Addressing performance traps in Bulk Insert/Layout Optimization

2021-12-06 Thread GitBox
hudi-bot commented on pull request #4234: URL: https://github.com/apache/hudi/pull/4234#issuecomment-987497932 ## CI report: * 45b092771c86e8955133699742a1f9bb9cbc18ab Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4234: [HUDI-2950][Stacked 4106] Addressing performance traps in Bulk Insert/Layout Optimization

2021-12-06 Thread GitBox
hudi-bot removed a comment on pull request #4234: URL: https://github.com/apache/hudi/pull/4234#issuecomment-987496885 ## CI report: * 45b092771c86e8955133699742a1f9bb9cbc18ab UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4234: [HUDI-2950][Stacked 4106] Addressing performance traps in Bulk Insert/Layout Optimization

2021-12-06 Thread GitBox
hudi-bot commented on pull request #4234: URL: https://github.com/apache/hudi/pull/4234#issuecomment-987496885 ## CI report: * 45b092771c86e8955133699742a1f9bb9cbc18ab UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[GitHub] [hudi] XuQianJin-Stars commented on a change in pull request #4193: [HUDI-2915] Fix field not found error for sparksql

2021-12-06 Thread GitBox
XuQianJin-Stars commented on a change in pull request #4193: URL: https://github.com/apache/hudi/pull/4193#discussion_r763568655 ## File path: hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java ## @@ -454,6 +457,10 @@ public static String getNestedFieldValAsSt

[GitHub] [hudi] hudi-bot removed a comment on pull request #4175: [HUDI-2883] Refactor hive sync tool / config to use reflection and standardize configs

2021-12-06 Thread GitBox
hudi-bot removed a comment on pull request #4175: URL: https://github.com/apache/hudi/pull/4175#issuecomment-987471304 ## CI report: * 92baf390f22bc9578344d8933621ff0e3778b007 UNKNOWN * 9c13f283384d94ed7b767b1dd89c8ccfe2d04d93 Azure: [SUCCESS](https://dev.azure.com/apache-hudi

[GitHub] [hudi] hudi-bot commented on pull request #4175: [HUDI-2883] Refactor hive sync tool / config to use reflection and standardize configs

2021-12-06 Thread GitBox
hudi-bot commented on pull request #4175: URL: https://github.com/apache/hudi/pull/4175#issuecomment-987495716 ## CI report: * 92baf390f22bc9578344d8933621ff0e3778b007 UNKNOWN * 4b4180032db7f8e216282a239e75e5f0593f354e UNKNOWN * da0912c93d8c6bc2e5a34aa4c480ef5b9d723b5a Azure

[jira] [Updated] (HUDI-2950) Address high small objects churn in Bulk Insert/Layout Optimization

2021-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2950: - Labels: pull-request-available (was: ) > Address high small objects churn in Bulk Insert/Layout O

[GitHub] [hudi] alexeykudinkin opened a new pull request #4234: [HUDI-2950][Stacked 4106] Addressing performance traps in Bulk Insert/Layout Optimization

2021-12-06 Thread GitBox
alexeykudinkin opened a new pull request #4234: URL: https://github.com/apache/hudi/pull/4234 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

[hudi] branch master updated: [MINOR] Remove redundant and conflicting spark-hive dependency (#4228)

2021-12-06 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 6dab307 [MINOR] Remove redundant and conflictin

[GitHub] [hudi] xushiyan merged pull request #4228: [MINOR] Remove redundant and conflicting spark-hive dependency

2021-12-06 Thread GitBox
xushiyan merged pull request #4228: URL: https://github.com/apache/hudi/pull/4228 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr

[GitHub] [hudi] leesf merged pull request #4231: [MINOR] Update committer list

2021-12-06 Thread GitBox
leesf merged pull request #4231: URL: https://github.com/apache/hudi/pull/4231 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[hudi] branch asf-site updated: [MINOR] Update committer list (#4231)

2021-12-06 Thread leesf
This is an automated email from the ASF dual-hosted git repository. leesf pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new c372a6d [MINOR] Update committer list (#4231)

[jira] [Updated] (HUDI-2950) Address high small objects churn in Bulk Insert/Layout Optimization

2021-12-06 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-2950: -- Description: Based on findings in HUDI-2949, following needs to be addressed to reduce pressure

[jira] [Updated] (HUDI-2950) Address high small objects churn in Bulk Insert/Layout Optimization

2021-12-06 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-2950: -- Status: Patch Available (was: In Progress) > Address high small objects churn in Bulk Insert/La

[jira] [Assigned] (HUDI-2950) Address high small objects churn in Bulk Insert/Layout Optimization

2021-12-06 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin reassigned HUDI-2950: - Assignee: Alexey Kudinkin > Address high small objects churn in Bulk Insert/Layout Optimi

[jira] [Updated] (HUDI-2950) Address high small objects churn during Bulk Insert

2021-12-06 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-2950: -- Description: Based on findings in HUDI-2949, following needs to be addressed to reduce pressure

[jira] [Updated] (HUDI-2950) Address high small objects churn in Bulk Insert/Layout Optimization

2021-12-06 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-2950: -- Summary: Address high small objects churn in Bulk Insert/Layout Optimization (was: Address high

[jira] [Created] (HUDI-2950) Address high small objects churn during Bulk Insert

2021-12-06 Thread Alexey Kudinkin (Jira)
Alexey Kudinkin created HUDI-2950: - Summary: Address high small objects churn during Bulk Insert Key: HUDI-2950 URL: https://issues.apache.org/jira/browse/HUDI-2950 Project: Apache Hudi Issue

[GitHub] [hudi] kywe665 commented on a change in pull request #4226: [HUDI-2821] - Docs for Metadata Table

2021-12-06 Thread GitBox
kywe665 commented on a change in pull request #4226: URL: https://github.com/apache/hudi/pull/4226#discussion_r763559035 ## File path: website/docs/metadata.md ## @@ -0,0 +1,34 @@ +--- +title: Metadata Table +keywords: [ hudi, metadata, S3 file listings] +--- + +## Motivation f

[jira] [Resolved] (HUDI-2949) Benchmark Clustering performance

2021-12-06 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin resolved HUDI-2949. --- > Benchmark Clustering performance > > > Key: HUD

[jira] [Updated] (HUDI-2949) Benchmark Clustering performance

2021-12-06 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-2949: -- Description: These observations are from running Layout Optimization (Clustering) on a [small A

[jira] [Updated] (HUDI-2949) Benchmark Clustering performance

2021-12-06 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-2949: -- Description: These observations are from running Layout Optimization (Clustering) on a [small A

[jira] [Assigned] (HUDI-2949) Benchmark Clustering performance

2021-12-06 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin reassigned HUDI-2949: - Assignee: Alexey Kudinkin > Benchmark Clustering performance > --

[jira] [Created] (HUDI-2949) Benchmark Clustering performance

2021-12-06 Thread Alexey Kudinkin (Jira)
Alexey Kudinkin created HUDI-2949: - Summary: Benchmark Clustering performance Key: HUDI-2949 URL: https://issues.apache.org/jira/browse/HUDI-2949 Project: Apache Hudi Issue Type: Sub-task

[jira] [Created] (HUDI-2948) [UMBRELLA] Hudi Clustering Performance

2021-12-06 Thread Alexey Kudinkin (Jira)
Alexey Kudinkin created HUDI-2948: - Summary: [UMBRELLA] Hudi Clustering Performance Key: HUDI-2948 URL: https://issues.apache.org/jira/browse/HUDI-2948 Project: Apache Hudi Issue Type: Task

[jira] [Updated] (HUDI-2948) [UMBRELLA] Hudi Clustering Performance

2021-12-06 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-2948: -- Fix Version/s: 0.11.0 > [UMBRELLA] Hudi Clustering Performance > ---

[jira] [Updated] (HUDI-2948) [UMBRELLA] Hudi Clustering Performance

2021-12-06 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-2948: -- Priority: Critical (was: Major) > [UMBRELLA] Hudi Clustering Performance >

[jira] [Assigned] (HUDI-2948) [UMBRELLA] Hudi Clustering Performance

2021-12-06 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin reassigned HUDI-2948: - Assignee: Alexey Kudinkin > [UMBRELLA] Hudi Clustering Performance >

[GitHub] [hudi] hudi-bot removed a comment on pull request #4175: [HUDI-2883] Refactor hive sync tool / config to use reflection and standardize configs

2021-12-06 Thread GitBox
hudi-bot removed a comment on pull request #4175: URL: https://github.com/apache/hudi/pull/4175#issuecomment-987426625 ## CI report: * 92baf390f22bc9578344d8933621ff0e3778b007 UNKNOWN * 9c13f283384d94ed7b767b1dd89c8ccfe2d04d93 Azure: [SUCCESS](https://dev.azure.com/apache-hudi

[GitHub] [hudi] hudi-bot commented on pull request #4175: [HUDI-2883] Refactor hive sync tool / config to use reflection and standardize configs

2021-12-06 Thread GitBox
hudi-bot commented on pull request #4175: URL: https://github.com/apache/hudi/pull/4175#issuecomment-987471304 ## CI report: * 92baf390f22bc9578344d8933621ff0e3778b007 UNKNOWN * 9c13f283384d94ed7b767b1dd89c8ccfe2d04d93 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/

[jira] [Updated] (HUDI-1834) Please delete old releases from mirroring system

2021-12-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-1834: -- Fix Version/s: 0.11.0 (was: 0.10.0) > Please delete old releases

[jira] [Commented] (HUDI-1834) Please delete old releases from mirroring system

2021-12-06 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17454323#comment-17454323 ] sivabalan narayanan commented on HUDI-1834: --- Archived everything from 0.5.1 unti

svn commit: r51226 - /release/hudi/0.6.0/

2021-12-06 Thread sivabalan
Author: sivabalan Date: Tue Dec 7 01:11:58 2021 New Revision: 51226 Log: Archiving old release 0.6.0 Removed: release/hudi/0.6.0/

svn commit: r51225 - /release/hudi/0.5.3/

2021-12-06 Thread sivabalan
Author: sivabalan Date: Tue Dec 7 01:11:47 2021 New Revision: 51225 Log: Archiving old release 0.5.3 Removed: release/hudi/0.5.3/

svn commit: r51224 - /release/hudi/0.5.2-incubating/

2021-12-06 Thread sivabalan
Author: sivabalan Date: Tue Dec 7 01:11:36 2021 New Revision: 51224 Log: Archiving old release 0.5.2 Removed: release/hudi/0.5.2-incubating/

svn commit: r51223 - /release/hudi/0.5.1-incubating/

2021-12-06 Thread sivabalan
Author: sivabalan Date: Tue Dec 7 01:11:17 2021 New Revision: 51223 Log: Archiving old release 0.5.1 Removed: release/hudi/0.5.1-incubating/

svn commit: r51222 - /release/hudi/0.5.0-incubating/

2021-12-06 Thread sivabalan
Author: sivabalan Date: Tue Dec 7 01:11:05 2021 New Revision: 51222 Log: Archiving old release 0.5.0 Removed: release/hudi/0.5.0-incubating/

svn commit: r51221 - /dev/hudi/hudi-0.6.0-rc1/

2021-12-06 Thread sivabalan
Author: sivabalan Date: Tue Dec 7 01:10:08 2021 New Revision: 51221 Log: Archiving old release Removed: dev/hudi/hudi-0.6.0-rc1/

svn commit: r51220 - /dev/hudi/hudi-0.6.0/

2021-12-06 Thread sivabalan
Author: sivabalan Date: Tue Dec 7 01:09:59 2021 New Revision: 51220 Log: Archiving old release Removed: dev/hudi/hudi-0.6.0/

svn commit: r51219 - /dev/hudi/hudi-0.5.3-rc2/

2021-12-06 Thread sivabalan
Author: sivabalan Date: Tue Dec 7 01:09:44 2021 New Revision: 51219 Log: Archiving old release Removed: dev/hudi/hudi-0.5.3-rc2/

svn commit: r51218 - /dev/hudi/hudi-0.5.3-rc1/

2021-12-06 Thread sivabalan
Author: sivabalan Date: Tue Dec 7 01:09:39 2021 New Revision: 51218 Log: Archiving old release Removed: dev/hudi/hudi-0.5.3-rc1/

svn commit: r51217 - /dev/hudi/hudi-0.5.3/

2021-12-06 Thread sivabalan
Author: sivabalan Date: Tue Dec 7 01:09:28 2021 New Revision: 51217 Log: Archiving old release Removed: dev/hudi/hudi-0.5.3/

  1   2   3   4   >