[GitHub] [hudi] CodeCooker17 commented on issue #5327: [SUPPORT]Mor table hive synchronization supports more flexible configuration

2022-04-15 Thread GitBox
CodeCooker17 commented on issue #5327: URL: https://github.com/apache/hudi/issues/5327#issuecomment-1099904189 > Here we need to add some configuration of synchronization rules. Is there some solution design for synchronization rules now? In addition to the two points mentioned above,

[GitHub] [hudi] hudi-bot commented on pull request #5329: [HUDI-3886] Adding default null for some of the fields in col stats in MDT schema

2022-04-15 Thread GitBox
hudi-bot commented on PR #5329: URL: https://github.com/apache/hudi/pull/5329#issuecomment-1099918952 ## CI report: * f5bde1e0961619a6ba26d6d8221a68ec4e5d0395 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8078

[jira] [Created] (HUDI-3887) Spark query can not read the data changes which written by flink after the spark query connection created

2022-04-15 Thread chacha.tang (Jira)
chacha.tang created HUDI-3887: - Summary: Spark query can not read the data changes which written by flink after the spark query connection created Key: HUDI-3887 URL: https://issues.apache.org/jira/browse/HUDI-3887

[GitHub] [hudi] XuQianJin-Stars commented on issue #5327: [SUPPORT]Mor table hive synchronization supports more flexible configuration

2022-04-15 Thread GitBox
XuQianJin-Stars commented on issue #5327: URL: https://github.com/apache/hudi/issues/5327#issuecomment-1099967428 > > Here we need to add some configuration of synchronization rules. > > Is there some solution design for synchronization rules now? In addition to the two points mention

[GitHub] [hudi] Guanpx opened a new issue, #5330: [SUPPORT] [BUG] Duplicate fileID ??? from bucket ?? of partition found during the BucketStreamWriteFunction index bootstrap.

2022-04-15 Thread GitBox
Guanpx opened a new issue, #5330: URL: https://github.com/apache/hudi/issues/5330 **Describe the problem you faced** use flink1.13 ,bucket index , cow ,hudi-0.11.0(not latest) **To Reproduce** Steps to reproduce the behavior: 1. start flink job 2. cancel flink

[GitHub] [hudi] kasured commented on issue #5298: [SUPPORT] File is deleted during inline compaction on MOR table causing subsequent FileNotFoundException on a reader

2022-04-15 Thread GitBox
kasured commented on issue #5298: URL: https://github.com/apache/hudi/issues/5298#issuecomment-104068 Upon further investigation and after enabling additional logs on EMR, the deletion of the file during compaction is happening in the class org.apache.hudi.table.HoodieTable#reconcileAga

[GitHub] [hudi] Guanpx commented on issue #5330: [SUPPORT] [BUG] Duplicate fileID ??? from bucket ?? of partition found during the BucketStreamWriteFunction index bootstrap.

2022-04-15 Thread GitBox
Guanpx commented on issue #5330: URL: https://github.com/apache/hudi/issues/5330#issuecomment-1100019832 cc @danny0405 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

[jira] [Closed] (HUDI-3867) Disable Data Skipping by default in 0.11

2022-04-15 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-3867. Resolution: Fixed > Disable Data Skipping by default in 0.11 > > >

[jira] [Updated] (HUDI-3707) Fix deltastreamer test with schema provider and transformer enabled

2022-04-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3707: -- Status: In Progress (was: Open) > Fix deltastreamer test with schema provider and trans

[jira] [Assigned] (HUDI-3707) Fix deltastreamer test with schema provider and transformer enabled

2022-04-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-3707: - Assignee: sivabalan narayanan (was: Sagar Sumit) > Fix deltastreamer test with s

[jira] [Updated] (HUDI-3888) Triage drop partition col with CI

2022-04-15 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3888: - Status: In Progress (was: Open) > Triage drop partition col with CI > - >

[jira] [Updated] (HUDI-3888) Triage drop partition col with CI

2022-04-15 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3888: - Sprint: Hudi-Sprint-Apr-12 > Triage drop partition col with CI > - > >

[jira] [Created] (HUDI-3888) Triage drop partition col with CI

2022-04-15 Thread Raymond Xu (Jira)
Raymond Xu created HUDI-3888: Summary: Triage drop partition col with CI Key: HUDI-3888 URL: https://issues.apache.org/jira/browse/HUDI-3888 Project: Apache Hudi Issue Type: Task Compon

[jira] [Updated] (HUDI-3826) Make truncate partition use delete_partition operation

2022-04-15 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3826: - Reviewers: Alexey Kudinkin, Raymond Xu, sivabalan narayanan (was: Alexey Kudinkin, sivabalan narayanan)

[GitHub] [hudi] hudi-bot commented on pull request #5328: [WIP] Fix Bulk Insert to repartition the dataset based on Partition Path

2022-04-15 Thread GitBox
hudi-bot commented on PR #5328: URL: https://github.com/apache/hudi/pull/5328#issuecomment-1100190821 ## CI report: * 96b33942edf6a1d6d89361d2e056ed1c3a8d326b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8077

[GitHub] [hudi] hudi-bot commented on pull request #5328: [WIP] Fix Bulk Insert to repartition the dataset based on Partition Path

2022-04-15 Thread GitBox
hudi-bot commented on PR #5328: URL: https://github.com/apache/hudi/pull/5328#issuecomment-1100192710 ## CI report: * 96b33942edf6a1d6d89361d2e056ed1c3a8d326b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8077

[GitHub] [hudi] nsivabalan commented on pull request #5328: [WIP] Fix Bulk Insert to repartition the dataset based on Partition Path

2022-04-15 Thread GitBox
nsivabalan commented on PR #5328: URL: https://github.com/apache/hudi/pull/5328#issuecomment-1100196310 high level comment. I would prefer to introduce a new sort mode instead of fixing NONE. and add documentation around when to use which sort mode so that users are aware of diff sort modes

[GitHub] [hudi] hudi-bot commented on pull request #5328: [WIP] Fix Bulk Insert to repartition the dataset based on Partition Path

2022-04-15 Thread GitBox
hudi-bot commented on PR #5328: URL: https://github.com/apache/hudi/pull/5328#issuecomment-1100234074 ## CI report: * 6812e0065e1411107d7d53ad2997d02e7ce34d06 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8079

[jira] [Updated] (HUDI-3889) Do not validate table config if save mode is set to Overwrite

2022-04-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3889: -- Fix Version/s: 0.12.0 > Do not validate table config if save mode is set to Overwrite >

[jira] [Updated] (HUDI-3889) Do not validate table config if save mode is set to Overwrite

2022-04-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3889: -- Priority: Critical (was: Major) > Do not validate table config if save mode is set to O

[jira] [Created] (HUDI-3889) Do not validate table config if save mode is set to Overwrite

2022-04-15 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-3889: - Summary: Do not validate table config if save mode is set to Overwrite Key: HUDI-3889 URL: https://issues.apache.org/jira/browse/HUDI-3889 Project: Apache H

[GitHub] [hudi] yihua opened a new pull request, #5331: [MINOR] Add a medium-definition Hudi logo

2022-04-15 Thread GitBox
yihua opened a new pull request, #5331: URL: https://github.com/apache/hudi/pull/5331 ## What is the purpose of the pull request As above. ## Brief change log - Adds `website/static/assets/images/hudi-logo-medium.png`. ## Verify this pull request The websi

[jira] [Created] (HUDI-3890) Fix apache rat check to detect all missing license

2022-04-15 Thread Raymond Xu (Jira)
Raymond Xu created HUDI-3890: Summary: Fix apache rat check to detect all missing license Key: HUDI-3890 URL: https://issues.apache.org/jira/browse/HUDI-3890 Project: Apache Hudi Issue Type: Task

[jira] [Updated] (HUDI-3890) Fix apache rat check to detect all missing license

2022-04-15 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3890: - Fix Version/s: 0.12.0 > Fix apache rat check to detect all missing license > -

[jira] [Updated] (HUDI-3890) Fix apache rat check to detect all missing license

2022-04-15 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3890: - Priority: Critical (was: Major) > Fix apache rat check to detect all missing license > --

[jira] [Commented] (HUDI-3749) Run latest hudi w/ EMR spark and report to aws folks

2022-04-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17522919#comment-17522919 ] sivabalan narayanan commented on HUDI-3749: --- regular hive sync worked out of the

[jira] [Assigned] (HUDI-3749) Run latest hudi w/ EMR spark and report to aws folks

2022-04-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-3749: - Assignee: Udit Mehrotra (was: sivabalan narayanan) > Run latest hudi w/ EMR spar

[jira] [Updated] (HUDI-3749) Try out 0.11 hudi w/ EMR spark

2022-04-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3749: -- Summary: Try out 0.11 hudi w/ EMR spark (was: Run latest hudi w/ EMR spark and report

[jira] [Commented] (HUDI-3749) Run latest hudi w/ EMR spark and report to aws folks

2022-04-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17522920#comment-17522920 ] sivabalan narayanan commented on HUDI-3749: --- Handing it off to [~uditme] to take

[GitHub] [hudi] nsivabalan merged pull request #5288: [HUDI-3848] Fixing restore with cleaned up commits

2022-04-15 Thread GitBox
nsivabalan merged PR #5288: URL: https://github.com/apache/hudi/pull/5288 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apach

[hudi] branch master updated (9e8664f4d2 -> 57612c5c32)

2022-04-15 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 9e8664f4d2 [HOTFIX] add missing license (#5322) (#5324) add 57612c5c32 [HUDI-3848] Fixing restore with cleaned u

[GitHub] [hudi] nsivabalan merged pull request #5182: [MINOR] Fixing parquet reader iterator close

2022-04-15 Thread GitBox
nsivabalan merged PR #5182: URL: https://github.com/apache/hudi/pull/5182 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apach

[GitHub] [hudi] nsivabalan commented on pull request #5329: [HUDI-3886] Adding default null for some of the fields in col stats in MDT schema

2022-04-15 Thread GitBox
nsivabalan commented on PR #5329: URL: https://github.com/apache/hudi/pull/5329#issuecomment-1100289976 @alexeykudinkin : can you stamp this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[hudi] branch master updated (57612c5c32 -> e8ab915aff)

2022-04-15 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 57612c5c32 [HUDI-3848] Fixing restore with cleaned up commits (#5288) add e8ab915aff [MINOR] Removing invalid co

[GitHub] [hudi] alexeykudinkin commented on pull request #5329: [HUDI-3886] Adding default null for some of the fields in col stats in MDT schema

2022-04-15 Thread GitBox
alexeykudinkin commented on PR #5329: URL: https://github.com/apache/hudi/pull/5329#issuecomment-1100291410 @nsivabalan done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] nsivabalan commented on pull request #5319: [WIP] Adjusting `DeltaStreamer` shutdown sequence to avoid awaiting for 24h

2022-04-15 Thread GitBox
nsivabalan commented on PR #5319: URL: https://github.com/apache/hudi/pull/5319#issuecomment-1100291962 please create a jira and tag -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] nsivabalan commented on pull request #5292: [WIP] Upgrade to Hadoop 3.x Hive 3.x

2022-04-15 Thread GitBox
nsivabalan commented on PR #5292: URL: https://github.com/apache/hudi/pull/5292#issuecomment-1100295279 please prefix w/ right jira. I understand, its still WIP. but a gentle reminder. -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [hudi] nsivabalan merged pull request #5270: [HUDI-3835] Add UT for delete in java client

2022-04-15 Thread GitBox
nsivabalan merged PR #5270: URL: https://github.com/apache/hudi/pull/5270 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apach

[hudi] branch master updated: [HUDI-3835] Add UT for delete in java client (#5270)

2022-04-15 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 99dd1cb6e6 [HUDI-3835] Add UT for delete in jav

[GitHub] [hudi] nsivabalan commented on pull request #5264: [HUDI-3818] encode bytes column value when generate HoodieKey

2022-04-15 Thread GitBox
nsivabalan commented on PR #5264: URL: https://github.com/apache/hudi/pull/5264#issuecomment-1100297837 generally record key, partition path and precombine should be comparable and so likely primitive types. wondering whats the use-case which demands byte[] to be chosen as a field for recor

[GitHub] [hudi] nsivabalan commented on pull request #5246: [HUDI-3813] [RFC-33] Schema Evolution Support DDL And DML Concurrency.

2022-04-15 Thread GitBox
nsivabalan commented on PR #5246: URL: https://github.com/apache/hudi/pull/5246#issuecomment-1100299905 @xushiyan : for now, I have assigned the PR to you. let me know if you can't take this up. I will find someone or I will take this up. -- This is an automated message from the Apac

[GitHub] [hudi] yihua opened a new pull request, #5332: [HUDI-3779] Update metadata table docs

2022-04-15 Thread GitBox
yihua opened a new pull request, #5332: URL: https://github.com/apache/hudi/pull/5332 ## What is the purpose of the pull request This PR updates metadata table docs with more detailed configurations and deployment considerations based on 0.11.0 release. ## Brief change log

[jira] [Updated] (HUDI-3779) Add docs regarding caveats for disabling and re-enabling MDT

2022-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3779: - Labels: pull-request-available (was: ) > Add docs regarding caveats for disabling and re-enabling

[jira] [Updated] (HUDI-3779) Add docs regarding caveats for disabling and re-enabling MDT

2022-04-15 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3779: Status: Patch Available (was: In Progress) > Add docs regarding caveats for disabling and re-enabling MDT >

[jira] [Updated] (HUDI-3779) Add docs regarding caveats for disabling and re-enabling MDT

2022-04-15 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3779: Status: In Progress (was: Open) > Add docs regarding caveats for disabling and re-enabling MDT > --

[GitHub] [hudi] nsivabalan closed pull request #5192: [WIP][DO_NOT_MERGE] Enable inline reading

2022-04-15 Thread GitBox
nsivabalan closed pull request #5192: [WIP][DO_NOT_MERGE] Enable inline reading URL: https://github.com/apache/hudi/pull/5192 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] nsivabalan commented on pull request #5177: [HUDI-3746][DO_NOT_MERGE] Test CI

2022-04-15 Thread GitBox
nsivabalan commented on PR #5177: URL: https://github.com/apache/hudi/pull/5177#issuecomment-1100334046 can we close this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] nsivabalan commented on pull request #5139: [WIP][HUDI-3579] Add timeline commands in hudi-cli

2022-04-15 Thread GitBox
nsivabalan commented on PR #5139: URL: https://github.com/apache/hudi/pull/5139#issuecomment-1100337466 @yihua : ping me once the patch is ready to be reviewed again -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hudi] nsivabalan commented on pull request #5111: [HUDI-3695] Add a ORC reader in HoodieBaseRelation

2022-04-15 Thread GitBox
nsivabalan commented on PR #5111: URL: https://github.com/apache/hudi/pull/5111#issuecomment-1100343028 @alexeykudinkin : can you follow up on the review when you get a chance. @miomiocat : can you rebase w/ latest master -- This is an automated message from the Apache Git Service. To r

[GitHub] [hudi] bhasudha opened a new pull request, #5333: [DOCS] update broken links

2022-04-15 Thread GitBox
bhasudha opened a new pull request, #5333: URL: https://github.com/apache/hudi/pull/5333 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpos

[GitHub] [hudi] kywe665 opened a new pull request, #5334: [MINOR] - updated external article list on Hudi docs

2022-04-15 Thread GitBox
kywe665 opened a new pull request, #5334: URL: https://github.com/apache/hudi/pull/5334 ## What is the purpose of the pull request updated the external articles for hudi docs ## Committer checklist - [X] Has a corresponding JIRA in PR title & commit - [X] Commit

[GitHub] [hudi] nsivabalan commented on pull request #5087: [HUDI-3614] [DO_NOT_MERGE]Replace List with HoodieData in HoodieFlink/JavaTable and commit executors

2022-04-15 Thread GitBox
nsivabalan commented on PR #5087: URL: https://github.com/apache/hudi/pull/5087#issuecomment-1100350369 @danny0405 : can you follow up on the patch when you get a chance. guess author is waiting for review follow up from you. a gentle reminder. -- This is an automated message from the Ap

[GitHub] [hudi] nsivabalan commented on pull request #5071: [HUDI-1881]: draft implementation for trigger based on data availability

2022-04-15 Thread GitBox
nsivabalan commented on PR #5071: URL: https://github.com/apache/hudi/pull/5071#issuecomment-1100367520 @pratyakshsharma : once the patch is ready, do ping me here. I can review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] nsivabalan commented on pull request #5057: [HUDI-3651] optimize the hoodie hive client and ddl executor code wit…

2022-04-15 Thread GitBox
nsivabalan commented on PR #5057: URL: https://github.com/apache/hudi/pull/5057#issuecomment-1100369298 @JerryYue-M : can you rebase w/ latest master -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hudi] nsivabalan commented on pull request #5057: [HUDI-3651] optimize the hoodie hive client and ddl executor code wit…

2022-04-15 Thread GitBox
nsivabalan commented on PR #5057: URL: https://github.com/apache/hudi/pull/5057#issuecomment-1100370034 @wangxianghu : can you review the patch when you get a chance -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hudi] nsivabalan commented on pull request #5064: [HUDI-3654] Initialize hudi metastore module.

2022-04-15 Thread GitBox
nsivabalan commented on PR #5064: URL: https://github.com/apache/hudi/pull/5064#issuecomment-1100372635 @xiarixiaoyao : can you review this when you get a chance. I have assigned it to myself as well. So, will try to review in a weeks time. -- This is an automated message from the Apache

[GitHub] [hudi] yihua merged pull request #5212: [MINOR] Fix typos in log4j-surefire.properties

2022-04-15 Thread GitBox
yihua merged PR #5212: URL: https://github.com/apache/hudi/pull/5212 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

[hudi] branch master updated: [MINOR] Fix typos in log4j-surefire.properties (#5212)

2022-04-15 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new b8e465fdfc [MINOR] Fix typos in log4j-surefire.prop

[GitHub] [hudi] yihua opened a new pull request, #5335: [MINOR] Fix docs build due to std-env

2022-04-15 Thread GitBox
yihua opened a new pull request, #5335: URL: https://github.com/apache/hudi/pull/5335 ## What is the purpose of the pull request This PR fixes the docs build due to the latest std-env 3.1.1 release. ## Brief change log - Uses "std-env" module from 3.0.1 instead in pac

[GitHub] [hudi] yihua commented on pull request #5335: [MINOR] Fix docs build due to std-env

2022-04-15 Thread GitBox
yihua commented on PR #5335: URL: https://github.com/apache/hudi/pull/5335#issuecomment-1100400894 cc @vingov @bhasudha -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[hudi] branch asf-site updated (805b893a71 -> d926276036)

2022-04-15 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git from 805b893a71 GitHub Actions build asf-site add d926276036 [MINOR] Fix docs build due to std-env (#5335) No new revi

[GitHub] [hudi] yihua merged pull request #5335: [MINOR] Fix docs build due to std-env

2022-04-15 Thread GitBox
yihua merged PR #5335: URL: https://github.com/apache/hudi/pull/5335 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

[hudi] branch asf-site updated (d926276036 -> ab49d9bcd8)

2022-04-15 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git from d926276036 [MINOR] Fix docs build due to std-env (#5335) add ab49d9bcd8 GitHub Actions build asf-site No new

[hudi] branch asf-site updated: [DOCS] Updated external article list on Hudi docs (#5334)

2022-04-15 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new ca6752b8a1 [DOCS] Updated external article list

[GitHub] [hudi] yihua merged pull request #5334: [MINOR] - updated external article list on Hudi docs

2022-04-15 Thread GitBox
yihua merged PR #5334: URL: https://github.com/apache/hudi/pull/5334 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

[jira] [Updated] (HUDI-3883) File-sizing issues when writing COW table to S3

2022-04-15 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3883: -- Fix Version/s: 0.12.0 > File-sizing issues when writing COW table to S3 > --

[GitHub] [hudi] yihua opened a new pull request, #5336: [DOCS] Add commit activity, twitter badgers, and Hudi logo in README

2022-04-15 Thread GitBox
yihua opened a new pull request, #5336: URL: https://github.com/apache/hudi/pull/5336 ## What is the purpose of the pull request This PR adds commit activity, twitter badgers, and Hudi logo in README. The medium-definition Hudi logo image is added to the Hudi site in #5331 .

[GitHub] [hudi] hudi-bot commented on pull request #5336: [DOCS] Add commit activity, twitter badgers, and Hudi logo in README

2022-04-15 Thread GitBox
hudi-bot commented on PR #5336: URL: https://github.com/apache/hudi/pull/5336#issuecomment-1100447825 ## CI report: * 2d1fc1b7ff81bff43152335b8135a31467c53674 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5336: [DOCS] Add commit activity, twitter badgers, and Hudi logo in README

2022-04-15 Thread GitBox
hudi-bot commented on PR #5336: URL: https://github.com/apache/hudi/pull/5336#issuecomment-1100448918 ## CI report: * 2d1fc1b7ff81bff43152335b8135a31467c53674 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8084

[GitHub] [hudi] hudi-bot commented on pull request #5336: [DOCS] Add commit activity, twitter badgers, and Hudi logo in README

2022-04-15 Thread GitBox
hudi-bot commented on PR #5336: URL: https://github.com/apache/hudi/pull/5336#issuecomment-1100475164 ## CI report: * 2d1fc1b7ff81bff43152335b8135a31467c53674 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8084

[jira] [Created] (HUDI-3891) Investigate Hudi vs Raw Parquet table discrepancy

2022-04-15 Thread Alexey Kudinkin (Jira)
Alexey Kudinkin created HUDI-3891: - Summary: Investigate Hudi vs Raw Parquet table discrepancy Key: HUDI-3891 URL: https://issues.apache.org/jira/browse/HUDI-3891 Project: Apache Hudi Issue T

[GitHub] [hudi] xiarixiaoyao commented on pull request #5064: [HUDI-3654] Initialize hudi metastore module.

2022-04-15 Thread GitBox
xiarixiaoyao commented on PR #5064: URL: https://github.com/apache/hudi/pull/5064#issuecomment-1100508616 @minihippo could you pls rebase the code and run azure again, thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [hudi] alexeykudinkin opened a new pull request, #5337: [HUDI-3891] Fixing files partitioning sequence for `BaseFileOnlyRelation`

2022-04-15 Thread GitBox
alexeykudinkin opened a new pull request, #5337: URL: https://github.com/apache/hudi/pull/5337 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

[jira] [Updated] (HUDI-3891) Investigate Hudi vs Raw Parquet table discrepancy

2022-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3891: - Labels: pull-request-available (was: ) > Investigate Hudi vs Raw Parquet table discrepancy >

[GitHub] [hudi] hudi-bot commented on pull request #5337: [HUDI-3891] Fixing files partitioning sequence for `BaseFileOnlyRelation`

2022-04-15 Thread GitBox
hudi-bot commented on PR #5337: URL: https://github.com/apache/hudi/pull/5337#issuecomment-1100509989 ## CI report: * 3da31d0812e520a29079c628c7a134bc66f066f1 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] danny0405 commented on pull request #5087: [HUDI-3614] [DO_NOT_MERGE]Replace List with HoodieData in HoodieFlink/JavaTable and commit executors

2022-04-15 Thread GitBox
danny0405 commented on PR #5087: URL: https://github.com/apache/hudi/pull/5087#issuecomment-1100510039 > @danny0405 : can you follow up on the patch when you get a chance. guess author is waiting for review follow up from you. a gentle reminder. I don't see there is any gains for curr

[GitHub] [hudi] danny0405 commented on pull request #5057: [HUDI-3651] optimize the hoodie hive client and ddl executor code wit…

2022-04-15 Thread GitBox
danny0405 commented on PR #5057: URL: https://github.com/apache/hudi/pull/5057#issuecomment-1100510685 @JerryYue-M You may need to rebase the code with latest master, would take a look soon ~ -- This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [hudi] hudi-bot commented on pull request #5337: [HUDI-3891] Fixing files partitioning sequence for `BaseFileOnlyRelation`

2022-04-15 Thread GitBox
hudi-bot commented on PR #5337: URL: https://github.com/apache/hudi/pull/5337#issuecomment-1100510773 ## CI report: * 3da31d0812e520a29079c628c7a134bc66f066f1 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8085

[GitHub] [hudi] hudi-bot commented on pull request #5337: [HUDI-3891] Fixing files partitioning sequence for `BaseFileOnlyRelation`

2022-04-15 Thread GitBox
hudi-bot commented on PR #5337: URL: https://github.com/apache/hudi/pull/5337#issuecomment-1100525225 ## CI report: * 3da31d0812e520a29079c628c7a134bc66f066f1 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8085

[GitHub] [hudi] nsivabalan commented on issue #5189: [SUPPORT] Multiple chaining of hudi tables via incremental source results in duplicate partition meta column

2022-04-15 Thread GitBox
nsivabalan commented on issue #5189: URL: https://github.com/apache/hudi/issues/5189#issuecomment-1100568175 @harsh1231 : in the mean time (until @bvaradar responds), can you investigate as to why we are encountering duplicate issue. -- This is an automated message from the Apache Git Se

[GitHub] [hudi] nsivabalan commented on issue #5198: [SUPPORT] Querying data genereated by TimestampBasedKeyGenerator failed to parse timestamp in EPOCHMILLISECONDS column to date format

2022-04-15 Thread GitBox
nsivabalan commented on issue #5198: URL: https://github.com/apache/hudi/issues/5198#issuecomment-1100568252 @babumahesh-koo : do you have any updates on this end -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] nsivabalan commented on issue #5211: [SUPPORT] Glob pattern to pick specific subfolders not working while reading in Spark

2022-04-15 Thread GitBox
nsivabalan commented on issue #5211: URL: https://github.com/apache/hudi/issues/5211#issuecomment-1100568451 So you want to read multiple hudi tables w/ one spark.read? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] nsivabalan commented on issue #5231: [SUPPORT] Inconsistent query result using GetLatestBaseFiles compared to Snapshot Query

2022-04-15 Thread GitBox
nsivabalan commented on issue #5231: URL: https://github.com/apache/hudi/issues/5231#issuecomment-1100568785 thanks @alexeykudinkin to find the root cause and fixing it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] nsivabalan closed issue #5231: [SUPPORT] Inconsistent query result using GetLatestBaseFiles compared to Snapshot Query

2022-04-15 Thread GitBox
nsivabalan closed issue #5231: [SUPPORT] Inconsistent query result using GetLatestBaseFiles compared to Snapshot Query URL: https://github.com/apache/hudi/issues/5231 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [hudi] nsivabalan commented on issue #5233: [SUPPORT] _hoodie_is_deleted not working for spark Datasource.

2022-04-15 Thread GitBox
nsivabalan commented on issue #5233: URL: https://github.com/apache/hudi/issues/5233#issuecomment-1100569034 did you set default value for "_hoodie_is_deleted" to null or false? can you post the schema for the table. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [hudi] nsivabalan commented on issue #5242: [SUPPORT] Hudi embedded timeline server in 0.9 vs 0.10 with `hoodie.embed.timeline.server.port`

2022-04-15 Thread GitBox
nsivabalan commented on issue #5242: URL: https://github.com/apache/hudi/issues/5242#issuecomment-1100570312 @yihua : timeline server port related issue. Can you chime in here please. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[GitHub] [hudi] nsivabalan commented on issue #5248: [QUESION] Should filter prop "hoodie.datasource.write.operation" when use spark sql create table?

2022-04-15 Thread GitBox
nsivabalan commented on issue #5248: URL: https://github.com/apache/hudi/issues/5248#issuecomment-1100570586 @XuQianJin-Stars : Can you file a tracking jira and follow up on the issue. seems like we need to fix this. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [hudi] nsivabalan commented on issue #5249: [SUPPORT] Deltastreamer job does not terminate on Kubernetes when hoodie.metrics.on=true

2022-04-15 Thread GitBox
nsivabalan commented on issue #5249: URL: https://github.com/apache/hudi/issues/5249#issuecomment-1100570805 @harsh1231 : Can you take a stab at this please. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [hudi] nsivabalan commented on issue #5258: [SUPPORT] Write hudi data throws NoSuchMethodError with spark v2.4.4 and hudi v0.10.1

2022-04-15 Thread GitBox
nsivabalan commented on issue #5258: URL: https://github.com/apache/hudi/issues/5258#issuecomment-1100571599 can you try w/ scala 11 bundle and let us know if it succeeds. hudi-spark-bundle_2.11-0.10.1.jar and for spark-avro, can you try setting it via `--packages org.apache.spark:spa

[GitHub] [hudi] nsivabalan commented on issue #5262: [SUPPORT] Deltastreamer Error upserting bucketType UPDATE for partition :0

2022-04-15 Thread GitBox
nsivabalan commented on issue #5262: URL: https://github.com/apache/hudi/issues/5262#issuecomment-1100572710 @stym06 : likely schema has changed. Can you inspect let us know if thats the case. related jira https://issues.apache.org/jira/browse/HUDI-1711 -- This is an automated message fro

[GitHub] [hudi] nsivabalan commented on issue #5281: [SUPPORT] .hoodie/hoodie.properties file can be deleted due to retention settings of cloud providers

2022-04-15 Thread GitBox
nsivabalan commented on issue #5281: URL: https://github.com/apache/hudi/issues/5281#issuecomment-1100574209 Interesting. whats your lifecycle policy btw? any objects that was never updated in the last X days to be deleted? -- This is an automated message from the Apache Git Service. To

[GitHub] [hudi] nsivabalan commented on issue #5291: [SUPPORT] How to use hudi-defaults.conf with Glue

2022-04-15 Thread GitBox
nsivabalan commented on issue #5291: URL: https://github.com/apache/hudi/issues/5291#issuecomment-1100576020 @zhedoubushishi : can you chime in here please. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] nsivabalan commented on issue #5301: [SUPPORT]Support Show Data Files Command Based on Call Procedure Command for Spark SQL

2022-04-15 Thread GitBox
nsivabalan commented on issue #5301: URL: https://github.com/apache/hudi/issues/5301#issuecomment-1100576398 @XuQianJin-Stars : Can you file a tracking jira and follow up please. and close out the github issue. -- This is an automated message from the Apache Git Service. To respon

[jira] [Created] (HUDI-3892) Add HoodieReadClient with java

2022-04-15 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-3892: - Summary: Add HoodieReadClient with java Key: HUDI-3892 URL: https://issues.apache.org/jira/browse/HUDI-3892 Project: Apache Hudi Issue Type: Task

[jira] [Updated] (HUDI-3892) Add HoodieReadClient with java

2022-04-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3892: -- Priority: Critical (was: Major) > Add HoodieReadClient with java >

[jira] [Updated] (HUDI-3892) Add HoodieReadClient with java

2022-04-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3892: -- Fix Version/s: 0.12.0 > Add HoodieReadClient with java > --

[GitHub] [hudi] nsivabalan commented on issue #5313: [SUPPORT] Do we have plan to support java reader for Hudi?

2022-04-15 Thread GitBox
nsivabalan commented on issue #5313: URL: https://github.com/apache/hudi/issues/5313#issuecomment-1100577200 cool. @simonsssu : I have created a tracking jira [here](https://issues.apache.org/jira/browse/HUDI-3892). Can you let me know your jira id. I can assign it to you. Also, this might

[GitHub] [hudi] nsivabalan commented on issue #5313: [SUPPORT] Do we have plan to support java reader for Hudi?

2022-04-15 Thread GitBox
nsivabalan commented on issue #5313: URL: https://github.com/apache/hudi/issues/5313#issuecomment-1100577433 @hangc0276 : We can definitely take this up. excited for hudi used as tiered storage :) As @simonsssu showed interest to work on it, I will coordinate w/ him/her and get this goi

[jira] [Updated] (HUDI-3892) Add HoodieReadClient with java

2022-04-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3892: -- Description: We might need a hoodie read client in java similar to the one we have for

[GitHub] [hudi] nsivabalan commented on issue #5326: [SUPPORT] prometheus metrics labels

2022-04-15 Thread GitBox
nsivabalan commented on issue #5326: URL: https://github.com/apache/hudi/issues/5326#issuecomment-1100577643 @harsh1231 : Can you chime in here please. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

  1   2   >