[jira] [Commented] (HUDI-3261) Query rt table by hive cli throw NoSuchMethodError

2022-01-18 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17477640#comment-17477640 ] Danny Chen commented on HUDI-3261: -- Thanks for the contribution, added. > Query rt table

[jira] [Assigned] (HUDI-3261) Query rt table by hive cli throw NoSuchMethodError

2022-01-18 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen reassigned HUDI-3261: Assignee: Echo Lee > Query rt table by hive cli throw NoSuchMethodError > -

[GitHub] [hudi] ChangbingChen commented on issue #4618: [SUPPORT] When querying a hudi table in hive, there have duplicated records.

2022-01-18 Thread GitBox
hive query, it wile scan those all parquet files? perhaps the newest one contains all records? ``` [yarn@x.x.x.x ~]$ hadoop fs -ls /hudi/mysql_table_sink_new/20220118 Found 9 items -rw-r--r-- 3 yarn supergroup 22309728 2022-01-18 15:22 /hudi/mysql_table_sink_new/20220118/.77dc5111-

[GitHub] [hudi] ChangbingChen removed a comment on issue #4618: [SUPPORT] When querying a hudi table in hive, there have duplicated records.

2022-01-18 Thread GitBox
adoop fs -ls /hudi/mysql_table_sink_new/20220118 Found 9 items -rw-r--r-- 3 yarn supergroup 22309728 2022-01-18 15:22 /hudi/mysql_table_sink_new/20220118/.77dc5111-0ed0-400c-9df3-84b254650ab5_20220118152035.log.1_0-1-0 -rw-r--r-- 3 yarn supergroup 26237250 2022-01-18 15:24

[GitHub] [hudi] ChangbingChen edited a comment on issue #4618: [SUPPORT] When querying a hudi table in hive, there have duplicated records.

2022-01-18 Thread GitBox
less then 128M. ``` [yarn@x.x.x ~]$ hadoop fs -ls /hudi/mysql_table_sink_new/20220118 Found 10 items -rw-r--r-- 3 yarn supergroup7157103 2022-01-18 11:17 /hudi/mysql_table_sink_new/20220118/.82f164fd-f97d-4691-b9c6-21bea2769be0_20220118111603.log.1_0-1-0 -rw-r--r-- 3 y

[GitHub] [hudi] hudi-bot removed a comment on pull request #4625: [HUDI-3263] Do not nullify members in HoodieTableFileSystemView#reset…

2022-01-18 Thread GitBox
hudi-bot removed a comment on pull request #4625: URL: https://github.com/apache/hudi/pull/4625#issuecomment-1015135059 ## CI report: * 31a00a1d995612cc616eab9df6c03b5fff87f098 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4625: [HUDI-3263] Do not nullify members in HoodieTableFileSystemView#reset…

2022-01-18 Thread GitBox
hudi-bot commented on pull request #4625: URL: https://github.com/apache/hudi/pull/4625#issuecomment-1015163939 ## CI report: * 31a00a1d995612cc616eab9df6c03b5fff87f098 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #4078: [HUDI-2833][Design] Merge small archive files instead of expanding indefinitely.

2022-01-18 Thread GitBox
hudi-bot commented on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1015168193 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * c36aac530d3350857fb01df858d0f26c123e5766 UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4078: [HUDI-2833][Design] Merge small archive files instead of expanding indefinitely.

2022-01-18 Thread GitBox
hudi-bot removed a comment on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1015132977 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * c36aac530d3350857fb01df858d0f26c123e5

[GitHub] [hudi] Guanpx commented on issue #4510: [SUPPORT] Impala query error

2022-01-18 Thread GitBox
Guanpx commented on issue #4510: URL: https://github.com/apache/hudi/issues/4510#issuecomment-1015177881 > @Guanpx : I don't have exp w/ impala. But was MOR querying working from impala for older versions of hudi and failing with 0.10.0 ? I think MOR does not work in any older versio

[GitHub] [hudi] danny0405 commented on pull request #4625: [HUDI-3263] Do not nullify members in HoodieTableFileSystemView#reset…

2022-01-18 Thread GitBox
danny0405 commented on pull request #4625: URL: https://github.com/apache/hudi/pull/4625#issuecomment-1015183371 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] zhangyue19921010 commented on pull request #4078: [HUDI-2833][Design] Merge small archive files instead of expanding indefinitely.

2022-01-18 Thread GitBox
zhangyue19921010 commented on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1015183981 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [hudi] hudi-bot commented on pull request #4078: [HUDI-2833][Design] Merge small archive files instead of expanding indefinitely.

2022-01-18 Thread GitBox
hudi-bot commented on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1015184441 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * c36aac530d3350857fb01df858d0f26c123e5766 UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4078: [HUDI-2833][Design] Merge small archive files instead of expanding indefinitely.

2022-01-18 Thread GitBox
hudi-bot removed a comment on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1015168193 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * c36aac530d3350857fb01df858d0f26c123e5

[GitHub] [hudi] hudi-bot commented on pull request #4625: [HUDI-3263] Do not nullify members in HoodieTableFileSystemView#reset…

2022-01-18 Thread GitBox
hudi-bot commented on pull request #4625: URL: https://github.com/apache/hudi/pull/4625#issuecomment-1015185058 ## CI report: * 31a00a1d995612cc616eab9df6c03b5fff87f098 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4625: [HUDI-3263] Do not nullify members in HoodieTableFileSystemView#reset…

2022-01-18 Thread GitBox
hudi-bot removed a comment on pull request #4625: URL: https://github.com/apache/hudi/pull/4625#issuecomment-1015163939 ## CI report: * 31a00a1d995612cc616eab9df6c03b5fff87f098 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4078: [HUDI-2833][Design] Merge small archive files instead of expanding indefinitely.

2022-01-18 Thread GitBox
hudi-bot removed a comment on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1015184441 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * c36aac530d3350857fb01df858d0f26c123e5

[GitHub] [hudi] hudi-bot commented on pull request #4078: [HUDI-2833][Design] Merge small archive files instead of expanding indefinitely.

2022-01-18 Thread GitBox
hudi-bot commented on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1015186808 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * c36aac530d3350857fb01df858d0f26c123e5766 UNKN

[GitHub] [hudi] peanut-chenzhong opened a new pull request #4626: [HUDI-1977] Fix Hudi CLI tempview query LOG issue

2022-01-18 Thread GitBox
peanut-chenzhong opened a new pull request #4626: URL: https://github.com/apache/hudi/pull/4626 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is th

[jira] [Updated] (HUDI-1977) Fix Hudi-CLI show table spark-sql

2022-01-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1977: - Labels: pull-request-available (was: ) > Fix Hudi-CLI show table spark-sql > ---

[GitHub] [hudi] hudi-bot commented on pull request #4626: [HUDI-1977] Fix Hudi CLI tempview query LOG issue

2022-01-18 Thread GitBox
hudi-bot commented on pull request #4626: URL: https://github.com/apache/hudi/pull/4626#issuecomment-1015189822 ## CI report: * 3860b11c8eb0823ffb1c8bcf23869b8c17c91df6 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] gunjdesai commented on issue #4437: [QUESTION] Example for CREATE TABLE on TRINO using HUDI

2022-01-18 Thread GitBox
gunjdesai commented on issue #4437: URL: https://github.com/apache/hudi/issues/4437#issuecomment-1015192029 Hi Team, any luck with this. I've tried asking this question on the slack channel for trino as well, but haven't got any luck there. -- This is an automated message from t

[GitHub] [hudi] hudi-bot removed a comment on pull request #4626: [HUDI-1977] Fix Hudi CLI tempview query LOG issue

2022-01-18 Thread GitBox
hudi-bot removed a comment on pull request #4626: URL: https://github.com/apache/hudi/pull/4626#issuecomment-1015189822 ## CI report: * 3860b11c8eb0823ffb1c8bcf23869b8c17c91df6 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4626: [HUDI-1977] Fix Hudi CLI tempview query LOG issue

2022-01-18 Thread GitBox
hudi-bot commented on pull request #4626: URL: https://github.com/apache/hudi/pull/4626#issuecomment-1015192443 ## CI report: * 3860b11c8eb0823ffb1c8bcf23869b8c17c91df6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] xushiyan commented on issue #4585: Target Schema cannot be set in MultiTableDeltaStreamer

2022-01-18 Thread GitBox
xushiyan commented on issue #4585: URL: https://github.com/apache/hudi/issues/4585#issuecomment-1015195895 @chrischnweiss So it makes sense to make the registry url more configurable. I would recommend you propose idea to improve this based on your use case. You can elaborate your idea her

[GitHub] [hudi] xiarixiaoyao commented on issue #4618: [SUPPORT] When querying a hudi table in hive, there have duplicated records.

2022-01-18 Thread GitBox
xiarixiaoyao commented on issue #4618: URL: https://github.com/apache/hudi/issues/4618#issuecomment-1015196785 @ChangbingChen sorry i forget one things, before you use hive to query hoodie table, do you have set inputformat, eg: set hive.input.format=org.apache.hudi.hadoop.hive.Hood

[jira] [Comment Edited] (HUDI-3222) On-call team to triage GH issues, PRs, and JIRAs

2022-01-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17477396#comment-17477396 ] Raymond Xu edited comment on HUDI-3222 at 1/18/22, 8:56 AM: h4

[GitHub] [hudi] danny0405 merged pull request #4624: [HUDI-3261] Query rt table by hive cli throw NoSuchMethodError

2022-01-18 Thread GitBox
danny0405 merged pull request #4624: URL: https://github.com/apache/hudi/pull/4624 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[jira] [Resolved] (HUDI-3261) Query rt table by hive cli throw NoSuchMethodError

2022-01-18 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-3261. -- > Query rt table by hive cli throw NoSuchMethodError > -- >

[jira] [Commented] (HUDI-3261) Query rt table by hive cli throw NoSuchMethodError

2022-01-18 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17477682#comment-17477682 ] Danny Chen commented on HUDI-3261: -- Fixed via master branch: 3b56320bd8f189786985fd44fcd4

[GitHub] [hudi] manojpec commented on a change in pull request #4523: [WIP][HUDI-3173] Add INDEX action type and corresponding commit metadata

2022-01-18 Thread GitBox
manojpec commented on a change in pull request #4523: URL: https://github.com/apache/hudi/pull/4523#discussion_r786532331 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieDefaultTimeline.java ## @@ -77,7 +77,9 @@ public void setInstants(List

[hudi] branch master updated (3d93e85 -> 3b56320)

2022-01-18 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 3d93e85 [MINOR] Minor improvement in JsonkafkaSource (#4620) add 3b56320 [HUDI-3261] Read rt table by hive cl

[GitHub] [hudi] pratyakshsharma commented on issue #4585: [Feature] Make schema registry configs more flexible with MultiTableDeltaStreamer

2022-01-18 Thread GitBox
pratyakshsharma commented on issue #4585: URL: https://github.com/apache/hudi/issues/4585#issuecomment-1015203741 > unfortunately our Kafka topic naming schema makes it impossible for us to use it this way. @chrischnweiss Are you trying to say you guys are using a subject naming str

[GitHub] [hudi] xiarixiaoyao commented on issue #4600: [SUPPORT]When hive queries Hudi data, the query path is wrong

2022-01-18 Thread GitBox
xiarixiaoyao commented on issue #4600: URL: https://github.com/apache/hudi/issues/4600#issuecomment-1015211429 @gubinjie if you donot want to modfiy hive code. could you pls trigger compaction for your table, one compaction done, parquet file will be created, and above problem should no

[GitHub] [hudi] xushiyan commented on a change in pull request #3745: [HUDI-2514] Add default hiveTableSerdeProperties for Spark SQL when sync Hive

2022-01-18 Thread GitBox
xushiyan commented on a change in pull request #3745: URL: https://github.com/apache/hudi/pull/3745#discussion_r786538718 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala ## @@ -366,50 +368,50 @@ object HoodieSparkSqlWrit

[jira] [Updated] (HUDI-2514) Add default hiveTableSerdeProperties for Spark SQL when sync Hive

2022-01-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2514: - Status: In Progress (was: Open) > Add default hiveTableSerdeProperties for Spark SQL when sync Hive > ---

[jira] [Updated] (HUDI-2514) Add default hiveTableSerdeProperties for Spark SQL when sync Hive

2022-01-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2514: - Status: Patch Available (was: In Progress) > Add default hiveTableSerdeProperties for Spark SQL when sync

[GitHub] [hudi] hudi-bot commented on pull request #4078: [HUDI-2833][Design] Merge small archive files instead of expanding indefinitely.

2022-01-18 Thread GitBox
hudi-bot commented on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1015236515 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * c36aac530d3350857fb01df858d0f26c123e5766 UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4078: [HUDI-2833][Design] Merge small archive files instead of expanding indefinitely.

2022-01-18 Thread GitBox
hudi-bot removed a comment on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1015186808 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * c36aac530d3350857fb01df858d0f26c123e5

[GitHub] [hudi] hudi-bot commented on pull request #4625: [HUDI-3263] Do not nullify members in HoodieTableFileSystemView#reset…

2022-01-18 Thread GitBox
hudi-bot commented on pull request #4625: URL: https://github.com/apache/hudi/pull/4625#issuecomment-1015237181 ## CI report: * 31a00a1d995612cc616eab9df6c03b5fff87f098 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4625: [HUDI-3263] Do not nullify members in HoodieTableFileSystemView#reset…

2022-01-18 Thread GitBox
hudi-bot removed a comment on pull request #4625: URL: https://github.com/apache/hudi/pull/4625#issuecomment-1015185058 ## CI report: * 31a00a1d995612cc616eab9df6c03b5fff87f098 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] danny0405 merged pull request #4625: [HUDI-3263] Do not nullify members in HoodieTableFileSystemView#reset…

2022-01-18 Thread GitBox
danny0405 merged pull request #4625: URL: https://github.com/apache/hudi/pull/4625 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[jira] [Resolved] (HUDI-3263) Do not nullify members in HoodieTableFileSystemView#resetViewState to avoid NPE

2022-01-18 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-3263. -- > Do not nullify members in HoodieTableFileSystemView#resetViewState to avoid > NPE > -

[hudi] branch master updated (3b56320 -> 45f054f)

2022-01-18 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 3b56320 [HUDI-3261] Read rt table by hive cli throw NoSuchMethodError (#4624) add 45f054f [HUDI-3263] Do not

[jira] [Commented] (HUDI-3263) Do not nullify members in HoodieTableFileSystemView#resetViewState to avoid NPE

2022-01-18 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17477717#comment-17477717 ] Danny Chen commented on HUDI-3263: -- Fixed via master branch: 45f054ffdef568e066a53c63c6e6

[GitHub] [hudi] ChangbingChen commented on issue #4618: [SUPPORT] When querying a hudi table in hive, there have duplicated records.

2022-01-18 Thread GitBox
ChangbingChen commented on issue #4618: URL: https://github.com/apache/hudi/issues/4618#issuecomment-1015244832 > @ChangbingChen sorry i forget one things, before you use hive to query hoodie table, do you have set inputformat, eg: set hive.input.format=org.apache.hudi.hadoop.hive.HoodieCo

[GitHub] [hudi] Guanpx opened a new issue #4510: [SUPPORT] Impala query error

2022-01-18 Thread GitBox
Guanpx opened a new issue #4510: URL: https://github.com/apache/hudi/issues/4510 **Describe the problem you faced** A clear and concise description of the problem. **To Reproduce** Steps to reproduce the behavior: 1. hudi sync hive 2. CREATE EXTERNAL IMPALA

[GitHub] [hudi] scxwhite commented on issue #4311: Duplicate Records in Merge on Read [SUPPORT]

2022-01-18 Thread GitBox
scxwhite commented on issue #4311: URL: https://github.com/apache/hudi/issues/4311#issuecomment-1015246379 I reproduced this problem using the following code. In the following code, I repeatedly update 1 pieces of data, but if I execute the following code more than 5 times, the program

[GitHub] [hudi] hudi-bot commented on pull request #4287: [DO NOT MERGE] 0.10.0 release patch for flink

2022-01-18 Thread GitBox
hudi-bot commented on pull request #4287: URL: https://github.com/apache/hudi/pull/4287#issuecomment-1015247003 ## CI report: * 5b7a535559d80359a3febc2d1a80bf9a8ac20cf9 UNKNOWN * 7231f68987a5f317f7d71a6485a4c2ea9f917a01 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4287: [DO NOT MERGE] 0.10.0 release patch for flink

2022-01-18 Thread GitBox
hudi-bot removed a comment on pull request #4287: URL: https://github.com/apache/hudi/pull/4287#issuecomment-1014450356 ## CI report: * 5b7a535559d80359a3febc2d1a80bf9a8ac20cf9 UNKNOWN * 7231f68987a5f317f7d71a6485a4c2ea9f917a01 Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] scxwhite commented on issue #4311: Duplicate Records in Merge on Read [SUPPORT]

2022-01-18 Thread GitBox
scxwhite commented on issue #4311: URL: https://github.com/apache/hudi/issues/4311#issuecomment-1015248403 In addition, my Hudi version is 0.9.0 and spark version is 3.0.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [hudi] melin opened a new issue #4627: [SUPPORT] Dremio integration

2022-01-18 Thread GitBox
melin opened a new issue #4627: URL: https://github.com/apache/hudi/issues/4627 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? - Join the mailing list to engage in conversations and get faster support at dev-subscr...@

[GitHub] [hudi] hudi-bot removed a comment on pull request #4626: [HUDI-1977] Fix Hudi CLI tempview query LOG issue

2022-01-18 Thread GitBox
hudi-bot removed a comment on pull request #4626: URL: https://github.com/apache/hudi/pull/4626#issuecomment-1015192443 ## CI report: * 3860b11c8eb0823ffb1c8bcf23869b8c17c91df6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4626: [HUDI-1977] Fix Hudi CLI tempview query LOG issue

2022-01-18 Thread GitBox
hudi-bot commented on pull request #4626: URL: https://github.com/apache/hudi/pull/4626#issuecomment-1015272593 ## CI report: * 3860b11c8eb0823ffb1c8bcf23869b8c17c91df6 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[jira] [Commented] (HUDI-2873) Support optimize data layout by sql and make the build more fast

2022-01-18 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17477743#comment-17477743 ] Tao Meng commented on HUDI-2873: [~shibei]  do you have wechat,  pls add me 1037817390 >

[GitHub] [hudi] hudi-bot commented on pull request #4287: [DO NOT MERGE] 0.10.0 release patch for flink

2022-01-18 Thread GitBox
hudi-bot commented on pull request #4287: URL: https://github.com/apache/hudi/pull/4287#issuecomment-1015274778 ## CI report: * 5b7a535559d80359a3febc2d1a80bf9a8ac20cf9 UNKNOWN * 7231f68987a5f317f7d71a6485a4c2ea9f917a01 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4287: [DO NOT MERGE] 0.10.0 release patch for flink

2022-01-18 Thread GitBox
hudi-bot removed a comment on pull request #4287: URL: https://github.com/apache/hudi/pull/4287#issuecomment-1015247003 ## CI report: * 5b7a535559d80359a3febc2d1a80bf9a8ac20cf9 UNKNOWN * 7231f68987a5f317f7d71a6485a4c2ea9f917a01 Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] gubinjie commented on issue #4600: [SUPPORT]When hive queries Hudi data, the query path is wrong

2022-01-18 Thread GitBox
gubinjie commented on issue #4600: URL: https://github.com/apache/hudi/issues/4600#issuecomment-1015286535 @xiarixiaoyao Thank you for your reply When I add a kafka connector, and then execute insert into 'hudi' select * from 'kafka', ('hudi' and 'kafka' are tables of connector type

[GitHub] [hudi] zhangyue19921010 removed a comment on pull request #4078: [HUDI-2833][Design] Merge small archive files instead of expanding indefinitely.

2022-01-18 Thread GitBox
zhangyue19921010 removed a comment on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1015183981 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [hudi] zhangyue19921010 commented on pull request #4078: [HUDI-2833][Design] Merge small archive files instead of expanding indefinitely.

2022-01-18 Thread GitBox
zhangyue19921010 commented on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1015294358 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [hudi] hudi-bot commented on pull request #4078: [HUDI-2833][Design] Merge small archive files instead of expanding indefinitely.

2022-01-18 Thread GitBox
hudi-bot commented on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1015296719 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * c36aac530d3350857fb01df858d0f26c123e5766 UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4078: [HUDI-2833][Design] Merge small archive files instead of expanding indefinitely.

2022-01-18 Thread GitBox
hudi-bot removed a comment on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1015236515 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * c36aac530d3350857fb01df858d0f26c123e5

[GitHub] [hudi] hudi-bot removed a comment on pull request #4287: [DO NOT MERGE] 0.10.0 release patch for flink

2022-01-18 Thread GitBox
hudi-bot removed a comment on pull request #4287: URL: https://github.com/apache/hudi/pull/4287#issuecomment-1015274778 ## CI report: * 5b7a535559d80359a3febc2d1a80bf9a8ac20cf9 UNKNOWN * 7231f68987a5f317f7d71a6485a4c2ea9f917a01 Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot commented on pull request #4287: [DO NOT MERGE] 0.10.0 release patch for flink

2022-01-18 Thread GitBox
hudi-bot commented on pull request #4287: URL: https://github.com/apache/hudi/pull/4287#issuecomment-1015311668 ## CI report: * 5b7a535559d80359a3febc2d1a80bf9a8ac20cf9 UNKNOWN * 952a154b1c656cd8e3c9c0df9fee313d3890d938 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] codope commented on a change in pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

2022-01-18 Thread GitBox
codope commented on a change in pull request #4352: URL: https://github.com/apache/hudi/pull/4352#discussion_r786556671 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/bloom/HoodieBloomIndex.java ## @@ -133,30 +144,89 @@ public HoodieBloomIndex

[GitHub] [hudi] dongkelun commented on a change in pull request #3745: [HUDI-2514] Add default hiveTableSerdeProperties for Spark SQL when sync Hive

2022-01-18 Thread GitBox
dongkelun commented on a change in pull request #3745: URL: https://github.com/apache/hudi/pull/3745#discussion_r786654476 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala ## @@ -366,50 +368,50 @@ object HoodieSparkSqlWri

[jira] [Updated] (HUDI-3222) On-call team to triage GH issues, PRs, and JIRAs

2022-01-18 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3222: - Reviewers: Raymond Xu, sivabalan narayanan > On-call team to triage GH issues, PRs, and JIRAs > --

[GitHub] [hudi] nsivabalan commented on issue #4552: [BUG] Data corrupted in the timestamp field to 1970-01-01 19:45:30.000 after subsequent upsert run

2022-01-18 Thread GitBox
nsivabalan commented on issue #4552: URL: https://github.com/apache/hudi/issues/4552#issuecomment-1015320225 Closing this one out since we know the root cause and have a solution. Feel free to re-open if you have more questions. would be happy to help. -- This is an automated message fr

[GitHub] [hudi] nsivabalan closed issue #4552: [BUG] Data corrupted in the timestamp field to 1970-01-01 19:45:30.000 after subsequent upsert run

2022-01-18 Thread GitBox
nsivabalan closed issue #4552: URL: https://github.com/apache/hudi/issues/4552 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[jira] [Created] (HUDI-3264) Make schema registry configs more flexible with MultiTableDeltaStreamer

2022-01-18 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-3264: - Summary: Make schema registry configs more flexible with MultiTableDeltaStreamer Key: HUDI-3264 URL: https://issues.apache.org/jira/browse/HUDI-3264 Project

[GitHub] [hudi] nsivabalan commented on issue #4585: [Feature] Make schema registry configs more flexible with MultiTableDeltaStreamer

2022-01-18 Thread GitBox
nsivabalan commented on issue #4585: URL: https://github.com/apache/hudi/issues/4585#issuecomment-1015323676 I have filed a [jira](https://issues.apache.org/jira/browse/HUDI-3264) on this end. @chrischnweiss : Feel free to update the jira w/ your suggestions. Even if you can't find cycles

[GitHub] [hudi] nsivabalan closed issue #4585: [Feature] Make schema registry configs more flexible with MultiTableDeltaStreamer

2022-01-18 Thread GitBox
nsivabalan closed issue #4585: URL: https://github.com/apache/hudi/issues/4585 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[GitHub] [hudi] nsivabalan commented on issue #4510: [SUPPORT] Impala query error

2022-01-18 Thread GitBox
nsivabalan commented on issue #4510: URL: https://github.com/apache/hudi/issues/4510#issuecomment-1015327025 WE already have a tracking jira to support MOR table type in Impala. If you are interested in working towards it, feel free to grab the jira and we can help with reviews if need be.

[GitHub] [hudi] nsivabalan closed issue #4510: [SUPPORT] Impala query error

2022-01-18 Thread GitBox
nsivabalan closed issue #4510: URL: https://github.com/apache/hudi/issues/4510 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[GitHub] [hudi] nsivabalan commented on issue #4457: [SUPPORT] Hudi archive stopped working

2022-01-18 Thread GitBox
nsivabalan commented on issue #4457: URL: https://github.com/apache/hudi/issues/4457#issuecomment-1015328411 @zuyanton : Do you have any updates for us. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [hudi] nsivabalan commented on issue #4456: [SUPPORT] MultiWriter w/ DynamoDB - Unable to acquire lock, lock object null

2022-01-18 Thread GitBox
nsivabalan commented on issue #4456: URL: https://github.com/apache/hudi/issues/4456#issuecomment-1015328893 @zhedoubushishi : When you get a chance, can you please follow up. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [hudi] nsivabalan closed issue #4439: [BUG] ROLLBACK meet Cannot use marker based rollback strategy on completed error

2022-01-18 Thread GitBox
nsivabalan closed issue #4439: URL: https://github.com/apache/hudi/issues/4439 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[GitHub] [hudi] nsivabalan commented on issue #4439: [BUG] ROLLBACK meet Cannot use marker based rollback strategy on completed error

2022-01-18 Thread GitBox
nsivabalan commented on issue #4439: URL: https://github.com/apache/hudi/issues/4439#issuecomment-1015329290 Feel free to re-open if you are looking for more assistance. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[GitHub] [hudi] nsivabalan commented on issue #4434: [SUPPORT]why are there many files under the Hoodie file?

2022-01-18 Thread GitBox
nsivabalan commented on issue #4434: URL: https://github.com/apache/hudi/issues/4434#issuecomment-1015329998 @tieke1121 : let us know if you have more questions /clarifications. If not, will close out the github issue. -- This is an automated message from the Apache Git Service. To resp

[GitHub] [hudi] nsivabalan commented on issue #4318: [SUPPORT] Duplicate records in COW table within same partition path

2022-01-18 Thread GitBox
nsivabalan commented on issue #4318: URL: https://github.com/apache/hudi/issues/4318#issuecomment-1015333164 Have updated instructions to access S3 via hudi-cli [here](https://hudi.apache.org/docs/next/cli#using-hudi-cli-in-s3). -- This is an automated message from the Apache Git Servic

[GitHub] [hudi] nsivabalan commented on issue #4318: [SUPPORT] Duplicate records in COW table within same partition path

2022-01-18 Thread GitBox
nsivabalan commented on issue #4318: URL: https://github.com/apache/hudi/issues/4318#issuecomment-1015334963 wrt duplicates, in general, a pair of partition path and record key is unique in hudi. If not, you need to use global index or non partitioned dataset if you wish to have unique

[GitHub] [hudi] stym06 commented on issue #4318: [SUPPORT] Duplicate records in COW table within same partition path

2022-01-18 Thread GitBox
stym06 commented on issue #4318: URL: https://github.com/apache/hudi/issues/4318#issuecomment-1015337273 @nsivabalan #3222 worked for me. Thanks for the help. We can close it out as the operation mode was INSERT and there were duplicate records coming in the Kafka topic as well, leading to

[GitHub] [hudi] stym06 closed issue #4318: [SUPPORT] Duplicate records in COW table within same partition path

2022-01-18 Thread GitBox
stym06 closed issue #4318: URL: https://github.com/apache/hudi/issues/4318 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hud

[GitHub] [hudi] hudi-bot removed a comment on pull request #4607: [HUDI-3161][RFC-46] Add Call Produce Command for Spark SQL

2022-01-18 Thread GitBox
hudi-bot removed a comment on pull request #4607: URL: https://github.com/apache/hudi/pull/4607#issuecomment-1013628091 ## CI report: * 9ddbc330d21f82188865a3a76af2b79a98101d3b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4607: [HUDI-3161][RFC-46] Add Call Produce Command for Spark SQL

2022-01-18 Thread GitBox
hudi-bot commented on pull request #4607: URL: https://github.com/apache/hudi/pull/4607#issuecomment-1015338194 ## CI report: * 9ddbc330d21f82188865a3a76af2b79a98101d3b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4607: [HUDI-3161][RFC-46] Add Call Produce Command for Spark SQL

2022-01-18 Thread GitBox
hudi-bot removed a comment on pull request #4607: URL: https://github.com/apache/hudi/pull/4607#issuecomment-1015338194 ## CI report: * 9ddbc330d21f82188865a3a76af2b79a98101d3b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4607: [HUDI-3161][RFC-46] Add Call Produce Command for Spark SQL

2022-01-18 Thread GitBox
hudi-bot commented on pull request #4607: URL: https://github.com/apache/hudi/pull/4607#issuecomment-1015340335 ## CI report: * 9ddbc330d21f82188865a3a76af2b79a98101d3b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] nsivabalan commented on issue #4241: [SUPPORT] Disaster Recovery (DR) Setup? Questions.

2022-01-18 Thread GitBox
nsivabalan commented on issue #4241: URL: https://github.com/apache/hudi/issues/4241#issuecomment-1015341274 We don't have any documentation as such. You need to directly use writeClient or go via hudi-cli. But here is how you can do savepoint and restore using hudi-cli ```

[GitHub] [hudi] nsivabalan edited a comment on issue #4241: [SUPPORT] Disaster Recovery (DR) Setup? Questions.

2022-01-18 Thread GitBox
nsivabalan edited a comment on issue #4241: URL: https://github.com/apache/hudi/issues/4241#issuecomment-1015341274 We don't have any documentation as such. You need to directly use writeClient or go via hudi-cli. Hudi-cli is the recommended way. But here is how you can do savepoint

[GitHub] [hudi] codope commented on issue #4541: [SUPPORT] NullPointerException while writing Bulk ingest table

2022-01-18 Thread GitBox
codope commented on issue #4541: URL: https://github.com/apache/hudi/issues/4541#issuecomment-1015341819 @nsivabalan Looks like AVRO_SCHEMA is not getting set in bulk insert mode. I couldn't find [similar logic](https://github.com/apache/hudi/blob/45f054ffdef568e066a53c63c6e6f8d2b1ee67ea/h

[jira] [Created] (HUDI-3265) Implement a custom serializer for the WriteStatus

2022-01-18 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-3265: - Summary: Implement a custom serializer for the WriteStatus Key: HUDI-3265 URL: https://issues.apache.org/jira/browse/HUDI-3265 Project: Apache Hudi

[jira] [Assigned] (HUDI-3265) Implement a custom serializer for the WriteStatus

2022-01-18 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-3265: - Assignee: Gary Li > Implement a custom serializer for the WriteStatus > -

[GitHub] [hudi] nsivabalan closed issue #4032: [SUPPORT] StreamWriteFunction WriteMetadataEvent serialization failed when WriteStatus structure changed

2022-01-18 Thread GitBox
nsivabalan closed issue #4032: URL: https://github.com/apache/hudi/issues/4032 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[GitHub] [hudi] nsivabalan commented on issue #4032: [SUPPORT] StreamWriteFunction WriteMetadataEvent serialization failed when WriteStatus structure changed

2022-01-18 Thread GitBox
nsivabalan commented on issue #4032: URL: https://github.com/apache/hudi/issues/4032#issuecomment-1015342387 Have filed a tracking [jira](https://issues.apache.org/jira/browse/HUDI-3265). will close the github issue. -- This is an automated message from the Apache Git Service. To respo

[GitHub] [hudi] nsivabalan commented on issue #3870: [SUPPORT] Hudi v0.8.0 Savepoint rollback failure

2022-01-18 Thread GitBox
nsivabalan commented on issue #3870: URL: https://github.com/apache/hudi/issues/3870#issuecomment-1015342619 @atharvai : hey do you have any updates for us. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] hudi-bot commented on pull request #4078: [HUDI-2833][Design] Merge small archive files instead of expanding indefinitely.

2022-01-18 Thread GitBox
hudi-bot commented on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1015344119 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * c36aac530d3350857fb01df858d0f26c123e5766 UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4078: [HUDI-2833][Design] Merge small archive files instead of expanding indefinitely.

2022-01-18 Thread GitBox
hudi-bot removed a comment on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1015296719 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * c36aac530d3350857fb01df858d0f26c123e5

[GitHub] [hudi] liujinhui1994 commented on issue #4311: Duplicate Records in Merge on Read [SUPPORT]

2022-01-18 Thread GitBox
liujinhui1994 commented on issue #4311: URL: https://github.com/apache/hudi/issues/4311#issuecomment-1015349498 Clustering does not currently support updates, this should be your problem. @scxwhite cc@nsivabalan -- This is an automated message from the Apache Git Service. To respond to

[jira] [Commented] (HUDI-1615) GH Issue 2515/ Failure to archive commits on row writer/delete paths

2022-01-18 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17477801#comment-17477801 ] sivabalan narayanan commented on HUDI-1615: --- [https://github.com/apache/hudi/pul

[GitHub] [hudi] nsivabalan commented on issue #4604: [SUPPORT] Archive functionality fails

2022-01-18 Thread GitBox
nsivabalan commented on issue #4604: URL: https://github.com/apache/hudi/issues/4604#issuecomment-1015351403 we have a related [issue](https://github.com/apache/hudi/pull/2653) reported earlier. Might help @XuQianJin-Stars triage it. -- This is an automated message from the Apache Git S

  1   2   3   4   5   6   7   >