[GitHub] [hudi] leesf merged pull request #4581: [MINOR] Optimize variable names and logs

2022-01-16 Thread GitBox
leesf merged pull request #4581: URL: https://github.com/apache/hudi/pull/4581 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[hudi] branch master updated (5e0171a -> 822230d)

2022-01-16 Thread leesf
This is an automated email from the ASF dual-hosted git repository. leesf pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 5e0171a [HUDI-3198] Improve Spark SQL create table from existing hudi table (#4584) add 822230d [MINOR] Optimize

[GitHub] [hudi] xiarixiaoyao commented on issue #4609: [SUPPORT] Got exception while using clustering with z-order

2022-01-16 Thread GitBox
xiarixiaoyao commented on issue #4609: URL: https://github.com/apache/hudi/issues/4609#issuecomment-1013834040 @ravs11 Thank you for your reply, could you pls give me the schema of the table first, i try to create some data to test. Of course, it would be better if there were some dummy

[GitHub] [hudi] XuQianJin-Stars opened a new pull request #4610: [HUDI-3251] Add HoodieFlinkSource for flink datastream api

2022-01-16 Thread GitBox
XuQianJin-Stars opened a new pull request #4610: URL: https://github.com/apache/hudi/pull/4610 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

[jira] [Updated] (HUDI-3253) preferred to use table's location

2022-01-16 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3253: - Sprint: Cont' improve - 2021/01/18 > preferred to use table's location >

[jira] [Updated] (HUDI-3251) Add HoodieFlinkSource for flink datastream api

2022-01-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3251: - Labels: pull-request-available (was: ) > Add HoodieFlinkSource for flink datastream api > ---

[jira] [Updated] (HUDI-3252) Avoid creating empty requestedReplaceCommit in the startCommit method

2022-01-16 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3252: - Sprint: Cont' improve - 2021/01/18 > Avoid creating empty requestedReplaceCommit in the startCommit metho

[jira] [Commented] (HUDI-3243) Fix flakiness in TestFSUtils

2022-01-16 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17476751#comment-17476751 ] Raymond Xu commented on HUDI-3243: -- [~codope] this should be closed?  > Fix flakiness in

[GitHub] [hudi] hudi-bot commented on pull request #4610: [HUDI-3251] Add HoodieFlinkSource for flink datastream api

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4610: URL: https://github.com/apache/hudi/pull/4610#issuecomment-1013835102 ## CI report: * a86f661cb5389577a0fed1339d0688c28204bb6f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot commented on pull request #4610: [HUDI-3251] Add HoodieFlinkSource for flink datastream api

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4610: URL: https://github.com/apache/hudi/pull/4610#issuecomment-1013835456 ## CI report: * a86f661cb5389577a0fed1339d0688c28204bb6f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4610: [HUDI-3251] Add HoodieFlinkSource for flink datastream api

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4610: URL: https://github.com/apache/hudi/pull/4610#issuecomment-1013835102 ## CI report: * a86f661cb5389577a0fed1339d0688c28204bb6f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] leesf merged pull request #4601: Fix class references to specific version dependent package

2022-01-16 Thread GitBox
leesf merged pull request #4601: URL: https://github.com/apache/hudi/pull/4601 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[hudi] branch master updated: [MINOR] Remove org.apache.directory.api.util.Strings import (#4601)

2022-01-16 Thread leesf
This is an automated email from the ASF dual-hosted git repository. leesf pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 28b3b6a [MINOR] Remove org.apache.directory.api.ut

[jira] [Resolved] (HUDI-3172) Refactor hudi existing modules to make more code reuse in V2 implementation

2022-01-16 Thread leesf (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf resolved HUDI-3172. - > Refactor hudi existing modules to make more code reuse in V2 implementation > --

[jira] [Closed] (HUDI-3172) Refactor hudi existing modules to make more code reuse in V2 implementation

2022-01-16 Thread leesf (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf closed HUDI-3172. --- > Refactor hudi existing modules to make more code reuse in V2 implementation > --

[jira] [Created] (HUDI-3254) Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread leesf (Jira)
leesf created HUDI-3254: --- Summary: Introduce HoodieCatalog to manage tables for Spark Datasource V2 Key: HUDI-3254 URL: https://issues.apache.org/jira/browse/HUDI-3254 Project: Apache Hudi Issue Type:

[GitHub] [hudi] xiarixiaoyao commented on issue #4593: [SUPPORT] Does Hudi support just column re-order?

2022-01-16 Thread GitBox
xiarixiaoyao commented on issue #4593: URL: https://github.com/apache/hudi/issues/4593#issuecomment-1013837875 @WTa-hash Hudi can't support column order adjustment very well at present, and rfc-33 is doing this We are testing rfc-33 in a production environment, once done will update

[GitHub] [hudi] leesf opened a new pull request #4611: [HUDI-3254] Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread GitBox
leesf opened a new pull request #4611: URL: https://github.com/apache/hudi/pull/4611 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the p

[jira] [Updated] (HUDI-3254) Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3254: - Labels: pull-request-available sev:normal (was: sev:normal) > Introduce HoodieCatalog to manage t

[GitHub] [hudi] hudi-bot commented on pull request #4611: [HUDI-3254] Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4611: URL: https://github.com/apache/hudi/pull/4611#issuecomment-1013838711 ## CI report: * f5e6315181eb95abc55bc6b1ee4f3488a60d65d0 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] xiarixiaoyao commented on issue #4600: [SUPPORT]When hive queries Hudi data, the query path is wrong

2022-01-16 Thread GitBox
xiarixiaoyao commented on issue #4600: URL: https://github.com/apache/hudi/issues/4600#issuecomment-1013838991 @gubinjie Are there only log files in your current Hudi table? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] hudi-bot removed a comment on pull request #4611: [HUDI-3254] Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4611: URL: https://github.com/apache/hudi/pull/4611#issuecomment-1013838711 ## CI report: * f5e6315181eb95abc55bc6b1ee4f3488a60d65d0 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4611: [HUDI-3254] Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4611: URL: https://github.com/apache/hudi/pull/4611#issuecomment-1013839138 ## CI report: * f5e6315181eb95abc55bc6b1ee4f3488a60d65d0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] xiarixiaoyao edited a comment on issue #4600: [SUPPORT]When hive queries Hudi data, the query path is wrong

2022-01-16 Thread GitBox
xiarixiaoyao edited a comment on issue #4600: URL: https://github.com/apache/hudi/issues/4600#issuecomment-1013838991 @gubinjie Are there only log files in your current Hudi table? now hive cannot read log only files, This is hive's problem. Hive will filter out all the files which s

[GitHub] [hudi] hudi-bot removed a comment on pull request #4611: [HUDI-3254] Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4611: URL: https://github.com/apache/hudi/pull/4611#issuecomment-1013839138 ## CI report: * f5e6315181eb95abc55bc6b1ee4f3488a60d65d0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4611: [HUDI-3254] Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4611: URL: https://github.com/apache/hudi/pull/4611#issuecomment-1013842745 ## CI report: * f5e6315181eb95abc55bc6b1ee4f3488a60d65d0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4611: [HUDI-3254] Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4611: URL: https://github.com/apache/hudi/pull/4611#issuecomment-1013842745 ## CI report: * f5e6315181eb95abc55bc6b1ee4f3488a60d65d0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4611: [HUDI-3254] Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4611: URL: https://github.com/apache/hudi/pull/4611#issuecomment-1013843123 ## CI report: * f5e6315181eb95abc55bc6b1ee4f3488a60d65d0 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #4610: [HUDI-3251] Add HoodieFlinkSource for flink datastream api

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4610: URL: https://github.com/apache/hudi/pull/4610#issuecomment-1013835456 ## CI report: * a86f661cb5389577a0fed1339d0688c28204bb6f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4610: [HUDI-3251] Add HoodieFlinkSource for flink datastream api

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4610: URL: https://github.com/apache/hudi/pull/4610#issuecomment-1013844291 ## CI report: * a86f661cb5389577a0fed1339d0688c28204bb6f Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] gubinjie commented on issue #4600: [SUPPORT]When hive queries Hudi data, the query path is wrong

2022-01-16 Thread GitBox
gubinjie commented on issue #4600: URL: https://github.com/apache/hudi/issues/4600#issuecomment-1013845360 @xiarixiaoyao @xushiyan thanks for the reply. 1. In my above steps, I have inserted six pieces of data through flink-sql, and these six pieces of data can also be found through f

[GitHub] [hudi] hudi-bot removed a comment on pull request #4611: [HUDI-3254] Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4611: URL: https://github.com/apache/hudi/pull/4611#issuecomment-1013843123 ## CI report: * f5e6315181eb95abc55bc6b1ee4f3488a60d65d0 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4611: [HUDI-3254] Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4611: URL: https://github.com/apache/hudi/pull/4611#issuecomment-1013848003 ## CI report: * f5e6315181eb95abc55bc6b1ee4f3488a60d65d0 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] xushiyan commented on issue #4506: [SUPPORT] Hive Sync fails silently with embedded derby hive metastore

2022-01-16 Thread GitBox
xushiyan commented on issue #4506: URL: https://github.com/apache/hudi/issues/4506#issuecomment-1013853416 @parisni can you try remove this setting? |spark.hadoop.javax.jdo.option.ConnectionURL |jdbc:derby:memory:myInMemDB;create=true| by default, hive cr

[GitHub] [hudi] hudi-bot removed a comment on pull request #4611: [HUDI-3254] Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4611: URL: https://github.com/apache/hudi/pull/4611#issuecomment-1013848003 ## CI report: * f5e6315181eb95abc55bc6b1ee4f3488a60d65d0 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4611: [HUDI-3254] Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4611: URL: https://github.com/apache/hudi/pull/4611#issuecomment-1013855071 ## CI report: * 27cba00d5daf148f99be78e72dade08229491d18 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4611: [HUDI-3254] Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4611: URL: https://github.com/apache/hudi/pull/4611#issuecomment-1013855071 ## CI report: * 27cba00d5daf148f99be78e72dade08229491d18 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4611: [HUDI-3254] Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4611: URL: https://github.com/apache/hudi/pull/4611#issuecomment-1013867484 ## CI report: * 27cba00d5daf148f99be78e72dade08229491d18 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[jira] [Created] (HUDI-3255) Add HoodieFlinkSink for flink datastream api

2022-01-16 Thread Forward Xu (Jira)
Forward Xu created HUDI-3255: Summary: Add HoodieFlinkSink for flink datastream api Key: HUDI-3255 URL: https://issues.apache.org/jira/browse/HUDI-3255 Project: Apache Hudi Issue Type: New Featur

[GitHub] [hudi] hudi-bot removed a comment on pull request #4611: [HUDI-3254] Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4611: URL: https://github.com/apache/hudi/pull/4611#issuecomment-1013867484 ## CI report: * 27cba00d5daf148f99be78e72dade08229491d18 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4611: [HUDI-3254] Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4611: URL: https://github.com/apache/hudi/pull/4611#issuecomment-1013867974 ## CI report: * 27cba00d5daf148f99be78e72dade08229491d18 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[jira] [Updated] (HUDI-3255) Add HoodieFlinkSink for flink datastream api

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3255: -- Component/s: flink > Add HoodieFlinkSink for flink datastream api >

[jira] [Updated] (HUDI-3252) Avoid creating empty requestedReplaceCommit in the startCommit method

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3252: -- Component/s: writer-core > Avoid creating empty requestedReplaceCommit in the startCommi

[jira] [Updated] (HUDI-3254) Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3254: -- Component/s: spark > Introduce HoodieCatalog to manage tables for Spark Datasource V2 >

[jira] [Updated] (HUDI-3248) Improve Hudi table services

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3248: -- Labels: archi (was: ) > Improve Hudi table services > --- > >

[jira] [Updated] (HUDI-3248) Improve Hudi table services

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3248: -- Labels: (was: archi) > Improve Hudi table services > --- > >

[jira] [Updated] (HUDI-3248) Improve Hudi table services

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3248: -- Component/s: archival > Improve Hudi table services > --- > >

[jira] [Updated] (HUDI-3250) Upgrade Presto version in docker setup and integ test

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3250: -- Component/s: trino-presto > Upgrade Presto version in docker setup and integ test >

[GitHub] [hudi] hudi-bot removed a comment on pull request #4611: [HUDI-3254] Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4611: URL: https://github.com/apache/hudi/pull/4611#issuecomment-1013867974 ## CI report: * 27cba00d5daf148f99be78e72dade08229491d18 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4611: [HUDI-3254] Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4611: URL: https://github.com/apache/hudi/pull/4611#issuecomment-1013868788 ## CI report: * 27cba00d5daf148f99be78e72dade08229491d18 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[jira] [Updated] (HUDI-3247) Support incremental queries in AbstractHoodieTableFileIndex

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3247: -- Component/s: incremental-query > Support incremental queries in AbstractHoodieTableFileI

[jira] [Updated] (HUDI-3246) Blog on Kafka Connect Sink for Hudi

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3246: -- Labels: kafka-connect (was: ) > Blog on Kafka Connect Sink for Hudi > -

[jira] [Updated] (HUDI-3245) Convert uppercase letters to lowercase in storage configs

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3245: -- Component/s: storage-management > Convert uppercase letters to lowercase in storage conf

[jira] [Updated] (HUDI-3246) Blog on Kafka Connect Sink for Hudi

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3246: -- Component/s: kafka-connect > Blog on Kafka Connect Sink for Hudi > -

[jira] [Updated] (HUDI-3243) Fix flakiness in TestFSUtils

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3243: -- Component/s: ci > Fix flakiness in TestFSUtils > > >

[jira] [Closed] (HUDI-3243) Fix flakiness in TestFSUtils

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-3243. - Assignee: Sagar Sumit Resolution: Fixed > Fix flakiness in TestFSUtils > ---

[GitHub] [hudi] hudi-bot removed a comment on pull request #4611: [HUDI-3254] Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4611: URL: https://github.com/apache/hudi/pull/4611#issuecomment-1013868788 ## CI report: * 27cba00d5daf148f99be78e72dade08229491d18 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4611: [HUDI-3254] Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4611: URL: https://github.com/apache/hudi/pull/4611#issuecomment-1013869160 ## CI report: * 27cba00d5daf148f99be78e72dade08229491d18 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[jira] [Updated] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3242: -- Labels: sev:critical (was: ) > Checkpoint 0 is ignored -Partial parquet file discovery

[jira] [Updated] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3242: -- Component/s: spark writer-core > Checkpoint 0 is ignored -Partial parqu

[jira] [Updated] (HUDI-3240) ALTER TABLE rename breaks with managed table in Spark 2.4

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3240: -- Component/s: spark-sql > ALTER TABLE rename breaks with managed table in Spark 2.4 > ---

[GitHub] [hudi] hudi-bot removed a comment on pull request #4611: [HUDI-3254] Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4611: URL: https://github.com/apache/hudi/pull/4611#issuecomment-1013869160 ## CI report: * 27cba00d5daf148f99be78e72dade08229491d18 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4611: [HUDI-3254] Introduce HoodieCatalog to manage tables for Spark Datasource V2

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4611: URL: https://github.com/apache/hudi/pull/4611#issuecomment-1013869592 ## CI report: * a5fecdc15707afd659887b025d1e1094543468e7 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[jira] [Updated] (HUDI-3239) Convert AbstractHoodieTableFileIndex to Java

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3239: -- Component/s: writer-core > Convert AbstractHoodieTableFileIndex to Java > --

[jira] [Updated] (HUDI-3236) ALTER TABLE COMMENT old comment gets reverted

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3236: -- Component/s: spark-sql > ALTER TABLE COMMENT old comment gets reverted > ---

[jira] [Updated] (HUDI-3237) ALTER TABLE column type change fails select query

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3237: -- Component/s: spark-sql > ALTER TABLE column type change fails select query > ---

[jira] [Updated] (HUDI-3235) Unit tests requiring hive service in setup failing

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3235: -- Fix Version/s: 0.11.0 > Unit tests requiring hive service in setup failing > ---

[jira] [Updated] (HUDI-3235) Unit tests requiring hive service in setup failing

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3235: -- Component/s: hive > Unit tests requiring hive service in setup failing > ---

[jira] [Closed] (HUDI-3235) Unit tests requiring hive service in setup failing

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-3235. - Resolution: Fixed > Unit tests requiring hive service in setup failing > -

[jira] [Updated] (HUDI-3232) support reload timeline Incrementally

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3232: -- Component/s: incremental-query > support reload timeline Incrementally > ---

[jira] [Commented] (HUDI-3235) Unit tests requiring hive service in setup failing

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17476778#comment-17476778 ] sivabalan narayanan commented on HUDI-3235: --- [~codope] : please re-open the tick

[jira] [Updated] (HUDI-3234) Fixing read of an empty table (with first commit failed) should return empty RDD

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3234: -- Component/s: writer-core > Fixing read of an empty table (with first commit failed) shou

[jira] [Updated] (HUDI-3234) Fixing read of an empty table (with first commit failed) should return empty RDD

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3234: -- Labels: sev:critical (was: ) > Fixing read of an empty table (with first commit failed)

[jira] [Updated] (HUDI-3232) support reload timeline Incrementally

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3232: -- Component/s: writer-core > support reload timeline Incrementally > -

[jira] [Updated] (HUDI-3226) Improve community on-call support tracker

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3226: -- Component/s: dev-experience > Improve community on-call support tracker > --

[jira] [Updated] (HUDI-3228) DataSource V2 review + discussions

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3228: -- Component/s: spark > DataSource V2 review + discussions > --

[jira] [Updated] (HUDI-3223) Triage GitHub issues (Siva)

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3223: -- Component/s: dev-experience > Triage GitHub issues (Siva) > ---

[jira] [Updated] (HUDI-3225) RFC for Async Metadata Index

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3225: -- Component/s: metadata > RFC for Async Metadata Index > > >

[jira] [Updated] (HUDI-3222) Triage GitHub issues (raymond)

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3222: -- Component/s: dev-experience > Triage GitHub issues (raymond) > -

[jira] [Updated] (HUDI-3221) Support querying a table as of a savepoint

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3221: -- Component/s: spark writer-core > Support querying a table as of a savep

[jira] [Updated] (HUDI-3221) Support querying a table as of a savepoint

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3221: -- Component/s: hive > Support querying a table as of a savepoint > ---

[jira] [Updated] (HUDI-3219) Summary of performance related issues that MetaIndex would address

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3219: -- Labels: metadata (was: ) > Summary of performance related issues that MetaIndex would a

[jira] [Updated] (HUDI-3218) Upgrade Avro to 1.10.2

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3218: -- Component/s: writer-core > Upgrade Avro to 1.10.2 > -- > >

[jira] [Updated] (HUDI-3217) Revisit Record Payload handling

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3217: -- Component/s: storage-management writer-core > Revisit Record Payload ha

[jira] [Updated] (HUDI-3216) Support timestamp with microseconds precision

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3216: -- Component/s: spark > Support timestamp with microseconds precision > ---

[jira] [Updated] (HUDI-3218) Upgrade Avro to 1.10.2

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3218: -- Component/s: storage-management > Upgrade Avro to 1.10.2 > -- > >

[jira] [Updated] (HUDI-3220) [UMBRELLA] Hudi Query Improvements

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3220: -- Component/s: hive > [UMBRELLA] Hudi Query Improvements > ---

[jira] [Updated] (HUDI-3211) RFC for Presto Hudi connector

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3211: -- Component/s: trino-presto > RFC for Presto Hudi connector > ---

[jira] [Updated] (HUDI-3212) Tuning merge small archive files

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3212: -- Component/s: writer-core > Tuning merge small archive files > --

[jira] [Updated] (HUDI-3207) Hudi Trino connector PR review

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3207: -- Component/s: trino-presto > Hudi Trino connector PR review > ---

[jira] [Updated] (HUDI-3208) Come up with rollout plan for enabling metadata table by default in 0.11

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3208: -- Component/s: metadata > Come up with rollout plan for enabling metadata table by default

[jira] [Updated] (HUDI-3202) Add keygen to support partition discovery

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3202: -- Component/s: spark > Add keygen to support partition discovery > ---

[jira] [Updated] (HUDI-3201) Make partition auto discovery configurable

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3201: -- Component/s: spark > Make partition auto discovery configurable > --

[jira] [Updated] (HUDI-3203) Meta bloom index should use the bloom filter type property to construct back the bloom filter instant

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3203: -- Component/s: metadata > Meta bloom index should use the bloom filter type property to co

[jira] [Updated] (HUDI-3200) File Index config affects partition fields shown in printSchema results

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3200: -- Component/s: spark writer-core > File Index config affects partition fi

[jira] [Updated] (HUDI-3205) Test savepoint and restore for MOR table with clustering

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3205: -- Component/s: writer-core > Test savepoint and restore for MOR table with clustering > --

[jira] [Updated] (HUDI-3202) Add keygen to support partition discovery

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3202: -- Component/s: writer-core > Add keygen to support partition discovery > -

[GitHub] [hudi] ravs11 commented on issue #4609: [SUPPORT] Got exception while using clustering with z-order

2022-01-16 Thread GitBox
ravs11 commented on issue #4609: URL: https://github.com/apache/hudi/issues/4609#issuecomment-1013871440 Hi @xiarixiaoyao , let me try to share some more details on this. **Hudi Table Schema** ``` CREATE TABLE dev.hudi_z_order_test ( product_id INT, product_name STRING,

[jira] [Updated] (HUDI-3199) Hive sync config unification

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3199: -- Component/s: hive > Hive sync config unification > - > >

[jira] [Updated] (HUDI-3194) Fix invisible writes(commits) during compaction (HoodieParquetRealtimeInputFormat)

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3194: -- Component/s: compaction writer-core > Fix invisible writes(commits) dur

  1   2   3   4   >