[GitHub] [hudi] YannByron commented on pull request #4270: [HUDI-2811] Support Spark 3.2

YannByron commented on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-991526869 @leesf @xushiyan please help to review this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [hudi] hudi-bot edited a comment on pull request #4278: [HUDI-2906] Add a repair util to clean up dangling data and log files

hudi-bot edited a comment on pull request #4278: URL: https://github.com/apache/hudi/pull/4278#issuecomment-991468392 ## CI report: * e8c56862a65de258d657f029cd15466f7e4e41f7 UNKNOWN * 432c2aff71bf918d22a3c6e81f23b11f5297d3b0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-

[GitHub] [hudi] yihua merged pull request #4278: [HUDI-2906] Add a repair util to clean up dangling data and log files

yihua merged pull request #4278: URL: https://github.com/apache/hudi/pull/4278 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[hudi] branch master updated (2dcb3f0 -> b5f05fd)

This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 2dcb3f0 [HUDI-2985] Shade jackson for hudi flink bundle jar (#4284) add b5f05fd [HUDI-2906] Add a repair util to

[GitHub] [hudi] hudi-bot commented on pull request #4285: [HUDI-2984] Implement #close for AbstractTableFileSystemView

hudi-bot commented on pull request #4285: URL: https://github.com/apache/hudi/pull/4285#issuecomment-991528462 ## CI report: * 7d18ceca4fb530966ba81e2c954bdf7885567839 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4285: [HUDI-2984] Implement #close for AbstractTableFileSystemView

hudi-bot removed a comment on pull request #4285: URL: https://github.com/apache/hudi/pull/4285#issuecomment-991502571 ## CI report: * 7d18ceca4fb530966ba81e2c954bdf7885567839 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] danny0405 merged pull request #4285: [HUDI-2984] Implement #close for AbstractTableFileSystemView

danny0405 merged pull request #4285: URL: https://github.com/apache/hudi/pull/4285 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[hudi] branch master updated: [HUDI-2984] Implement #close for AbstractTableFileSystemView (#4285)

This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 8dd0444 [HUDI-2984] Implement #close for Abstr

[jira] [Resolved] (HUDI-2984) Implement #close for AbstractTableFileSystemView

[ https://issues.apache.org/jira/browse/HUDI-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-2984. -- > Implement #close for AbstractTableFileSystemView > > >

[jira] [Commented] (HUDI-2984) Implement #close for AbstractTableFileSystemView

[ https://issues.apache.org/jira/browse/HUDI-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17457560#comment-17457560 ] Danny Chen commented on HUDI-2984: -- Fixed via master branch: 8dd0444ef96db4028bd28d5e4d86

[GitHub] [hudi] danny0405 opened a new pull request #4287: [DO NOT MERGE] 0.10.0 release patch for flink

danny0405 opened a new pull request #4287: URL: https://github.com/apache/hudi/pull/4287 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpo

[GitHub] [hudi] hudi-bot commented on pull request #4287: [DO NOT MERGE] 0.10.0 release patch for flink

hudi-bot commented on pull request #4287: URL: https://github.com/apache/hudi/pull/4287#issuecomment-991548169 ## CI report: * 8431c896cde1284a5a8068c8f34133984c165771 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[GitHub] [hudi] hudi-bot commented on pull request #4286: [WIP][HUDI-2955] Upgrade Hadoop to 3.3.1, Hive to 3.1.2, HBase to 2.4.8

hudi-bot commented on pull request #4286: URL: https://github.com/apache/hudi/pull/4286#issuecomment-991552638 ## CI report: * 4a459976c56d12c1beb46284862113b866bba284 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4286: [WIP][HUDI-2955] Upgrade Hadoop to 3.3.1, Hive to 3.1.2, HBase to 2.4.8

hudi-bot removed a comment on pull request #4286: URL: https://github.com/apache/hudi/pull/4286#issuecomment-991506999 ## CI report: * 4a459976c56d12c1beb46284862113b866bba284 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot removed a comment on pull request #4287: [DO NOT MERGE] 0.10.0 release patch for flink

hudi-bot removed a comment on pull request #4287: URL: https://github.com/apache/hudi/pull/4287#issuecomment-991548169 ## CI report: * 8431c896cde1284a5a8068c8f34133984c165771 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4287: [DO NOT MERGE] 0.10.0 release patch for flink

hudi-bot commented on pull request #4287: URL: https://github.com/apache/hudi/pull/4287#issuecomment-991552652 ## CI report: * 8431c896cde1284a5a8068c8f34133984c165771 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] YannByron commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

YannByron commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767130598 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/avro/HoodieAvroDeserializer.scala ## @@ -18,17 +18,25 @@ package org.a

[GitHub] [hudi] YannByron commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

YannByron commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767130805 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/Compaction.scala ## @@ -22,17 +22,37 @@ import

[GitHub] [hudi] YannByron commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

YannByron commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767130856 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/analysis/HoodieAnalysis.scala ## @@ -244,13 +253,19 @@ case class

[GitHub] [hudi] YannByron commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

YannByron commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767130865 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/analysis/HoodieAnalysis.scala ## @@ -244,13 +253,19 @@ case class

[GitHub] [hudi] YannByron commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

YannByron commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767130934 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/analysis/HoodieAnalysis.scala ## @@ -424,9 +439,9 @@ case class Ho

[GitHub] [hudi] YannByron commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

YannByron commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767131125 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/AlterHoodieTableAddColumnsCommand.scala ## @@ -84,12 +89,1

[GitHub] [hudi] YannByron commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

YannByron commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767131264 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/MergeIntoHoodieTableCommand.scala ## @@ -203,7 +207,11 @@

[GitHub] [hudi] YannByron commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

YannByron commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767131307 ## File path: hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/TestHoodieSparkSqlWriter.scala ## @@ -293,28 +293,26 @@ class TestHoodieSpar

[jira] [Created] (HUDI-2986) Deltastreamer continuous mode run into Too many open files exception

Raymond Xu created HUDI-2986: Summary: Deltastreamer continuous mode run into Too many open files exception Key: HUDI-2986 URL: https://issues.apache.org/jira/browse/HUDI-2986 Project: Apache Hudi

[jira] [Created] (HUDI-2987) event time not recorded in commit metadata when insert or bulk insert

Raymond Xu created HUDI-2987: Summary: event time not recorded in commit metadata when insert or bulk insert Key: HUDI-2987 URL: https://issues.apache.org/jira/browse/HUDI-2987 Project: Apache Hudi

[GitHub] [hudi] YannByron commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

YannByron commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767131694 ## File path: hudi-spark-datasource/hudi-spark3/src/main/scala/org/apache/spark/sql/execution/datasources/Spark3ParsePartitionUtil.scala ## @@ -16,24 +16,

[GitHub] [hudi] YannByron commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

YannByron commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767131125 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/AlterHoodieTableAddColumnsCommand.scala ## @@ -84,12 +89,1

[jira] [Created] (HUDI-2988) Add Event time configuration: latency adjustment

Raymond Xu created HUDI-2988: Summary: Add Event time configuration: latency adjustment Key: HUDI-2988 URL: https://issues.apache.org/jira/browse/HUDI-2988 Project: Apache Hudi Issue Type: Improv

[GitHub] [hudi] hudi-bot removed a comment on pull request #4287: [DO NOT MERGE] 0.10.0 release patch for flink

hudi-bot removed a comment on pull request #4287: URL: https://github.com/apache/hudi/pull/4287#issuecomment-991552652 ## CI report: * 8431c896cde1284a5a8068c8f34133984c165771 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4287: [DO NOT MERGE] 0.10.0 release patch for flink

hudi-bot commented on pull request #4287: URL: https://github.com/apache/hudi/pull/4287#issuecomment-991570858 ## CI report: * 8431c896cde1284a5a8068c8f34133984c165771 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[jira] [Created] (HUDI-2989) Hive sync to Glue tables not updating S3 location

Raymond Xu created HUDI-2989: Summary: Hive sync to Glue tables not updating S3 location Key: HUDI-2989 URL: https://issues.apache.org/jira/browse/HUDI-2989 Project: Apache Hudi Issue Type: Bug

[GitHub] [hudi] leesf commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

leesf commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767162488 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/avro/HoodieAvroDeserializer.scala ## @@ -18,17 +18,25 @@ package org.apach

[GitHub] [hudi] leesf commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

leesf commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767162581 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/Compaction.scala ## @@ -22,17 +22,37 @@ import org.

[GitHub] [hudi] leesf commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

leesf commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767162767 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/Compaction.scala ## @@ -22,17 +22,37 @@ import org.

[GitHub] [hudi] leesf commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

leesf commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767162840 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/analysis/HoodieAnalysis.scala ## @@ -180,11 +181,19 @@ case class Hood

[GitHub] [hudi] leesf commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

leesf commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767162840 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/analysis/HoodieAnalysis.scala ## @@ -180,11 +181,19 @@ case class Hood

[GitHub] [hudi] leesf commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

leesf commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767162840 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/analysis/HoodieAnalysis.scala ## @@ -180,11 +181,19 @@ case class Hood

[GitHub] [hudi] leesf commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

leesf commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767162910 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/analysis/HoodieAnalysis.scala ## @@ -244,13 +253,19 @@ case class Hood

[GitHub] [hudi] leesf commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

leesf commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767163001 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/analysis/HoodieAnalysis.scala ## @@ -244,13 +253,19 @@ case class Hood

[GitHub] [hudi] leesf commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

leesf commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767163040 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/analysis/HoodieAnalysis.scala ## @@ -244,13 +253,19 @@ case class Hood

[GitHub] [hudi] leesf commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

leesf commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767163040 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/analysis/HoodieAnalysis.scala ## @@ -244,13 +253,19 @@ case class Hood

[GitHub] [hudi] leesf commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

leesf commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767163296 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/AlterHoodieTableAddColumnsCommand.scala ## @@ -59,7 +64,7 @@ c

[GitHub] [hudi] leesf commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

leesf commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767163415 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/MergeIntoHoodieTableCommand.scala ## @@ -203,7 +207,11 @@ case

[GitHub] [hudi] york-yu-ctw commented on issue #4283: [SUPPORT] Data written by hudi 0.10.0 is not able be query by redshift

york-yu-ctw commented on issue #4283: URL: https://github.com/apache/hudi/issues/4283#issuecomment-991685485 I tired to change the logic in HoodieInstantTimeGenerator to create **second** instant timestamp and build the jar file, it works -- This is an automated message from the Apache

[GitHub] [hudi] york-yu-ctw commented on issue #4283: [SUPPORT] Data written by hudi 0.10.0 is not able be query by redshift

york-yu-ctw commented on issue #4283: URL: https://github.com/apache/hudi/issues/4283#issuecomment-991686979 I found some discussion in https://github.com/apache/hudi/pull/4024 As the contributor mentioned, the functions to parse timestamp are static, it may cause parameter parsi

[GitHub] [hudi] york-yu-ctw edited a comment on issue #4283: [SUPPORT] Data written by hudi 0.10.0 is not able be query by redshift

york-yu-ctw edited a comment on issue #4283: URL: https://github.com/apache/hudi/issues/4283#issuecomment-991686979 I found some discussion in https://github.com/apache/hudi/pull/4024 As the contributor mentioned, the functions to parse timestamp are static, if we are going to ha

[GitHub] [hudi] york-yu-ctw edited a comment on issue #4283: [SUPPORT] Data written by hudi 0.10.0 is not able be query by redshift

york-yu-ctw edited a comment on issue #4283: URL: https://github.com/apache/hudi/issues/4283#issuecomment-991685485 After I tired to change the logic in HoodieInstantTimeGenerator to create **second** instant timestamp, it works again -- This is an automated message from the Apache Git S

[GitHub] [hudi] nsivabalan commented on a change in pull request #4253: [HUDI-2958] Automatically set spark.sql.parquet.writelegacyformat, when using bulkinsert to insert data which contains decimalTy

nsivabalan commented on a change in pull request #4253: URL: https://github.com/apache/hudi/pull/4253#discussion_r767174406 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/io/storage/row/HoodieRowParquetWriteSupport.java ## @@ -46,13 +53,32 @@ publ

[GitHub] [hudi] nikita-sheremet-clearscale commented on issue #4267: [SUPPORT] Hudi partition values not getting reflected in Athena

nikita-sheremet-clearscale commented on issue #4267: URL: https://github.com/apache/hudi/issues/4267#issuecomment-991731335 I do not know how to fix this for Glue because it hides all nodes from management. But I know how to fix this error for EMR. The source article is - https://aws.am

[GitHub] [hudi] nikita-sheremet-clearscale edited a comment on issue #4267: [SUPPORT] Hudi partition values not getting reflected in Athena

nikita-sheremet-clearscale edited a comment on issue #4267: URL: https://github.com/apache/hudi/issues/4267#issuecomment-991731335 I do not know how to fix this for Glue because it hides all nodes from management. But I know how to fix this error for EMR. The source article is - https:/

[GitHub] [hudi] nikita-sheremet-clearscale opened a new issue #4288: [SUPPORT] Could you please clarify hoodie.datasource.hive_sync.mode config value in documentation?

nikita-sheremet-clearscale opened a new issue #4288: URL: https://github.com/apache/hudi/issues/4288 The config key `hoodie.datasource.hive_sync.mode` appeared in hudi 0.9.0 version. Maybe I miss something but I am not able to find this parameter in previous versions. How hudi worked witho

[GitHub] [hudi] xushiyan merged pull request #4289: [MINOR] Fix Spark SQL cases

xushiyan merged pull request #4289: URL: https://github.com/apache/hudi/pull/4289 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr

[hudi] branch asf-site updated: [MINOR] Fix Spark SQL cases (#4289)

This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 79c1733 [MINOR] Fix Spark SQL cases (#4289)

[GitHub] [hudi] YannByron commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

YannByron commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767213813 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/analysis/HoodieAnalysis.scala ## @@ -180,11 +181,19 @@ case class

[GitHub] [hudi] YannByron commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

YannByron commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767214963 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/analysis/HoodieAnalysis.scala ## @@ -244,13 +253,19 @@ case class

[GitHub] [hudi] YannByron commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

YannByron commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767216662 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/avro/HoodieAvroDeserializer.scala ## @@ -18,17 +18,25 @@ package org.a

[GitHub] [hudi] danny0405 opened a new pull request #4290: [minor] fix flink notification for streaming ingestion and quick start

danny0405 opened a new pull request #4290: URL: https://github.com/apache/hudi/pull/4290 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpo

[GitHub] [hudi] danny0405 merged pull request #4290: [minor] fix flink notification for streaming ingestion and quick start

danny0405 merged pull request #4290: URL: https://github.com/apache/hudi/pull/4290 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[hudi] branch asf-site updated: [minor] fix flink notification for streaming ingestion and quick start (#4290)

This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 48119f1 [minor] fix flink notification for

[GitHub] [hudi] YannByron commented on a change in pull request #4270: [HUDI-2811] Support Spark 3.2

YannByron commented on a change in pull request #4270: URL: https://github.com/apache/hudi/pull/4270#discussion_r767217435 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/Compaction.scala ## @@ -22,17 +22,37 @@ import

[GitHub] [hudi] nsivabalan commented on issue #4283: [SUPPORT] Data written by hudi 0.10.0 is not able be query by redshift

nsivabalan commented on issue #4283: URL: https://github.com/apache/hudi/issues/4283#issuecomment-991829623 @york-yu-ctw : Can you raise a support ticket against redshift? looks like hudi::ParsedFilename::IsValidCommitTimestamp has to be fixed when using 0.10.0. unfortunately, we did no

[GitHub] [hudi] nsivabalan commented on issue #4288: [SUPPORT] Could you please clarify hoodie.datasource.hive_sync.mode config value in documentation?

nsivabalan commented on issue #4288: URL: https://github.com/apache/hudi/issues/4288#issuecomment-991829873 @codope Can you chime in here. Also, @kywe665 : can we improve the docs if this is not yet covered. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] nsivabalan commented on issue #4233: [SUPPORT] Hudi parquet INT16 conversion loses LogicalType information

nsivabalan commented on issue #4233: URL: https://github.com/apache/hudi/issues/4233#issuecomment-991830526 thanks for updating with the resolution. I will add it to our faq to assist others in the community. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] nsivabalan commented on issue #4230: [SUPPORT] org.apache.hudi.exception.HoodieRemoteException: Failed to create marker file

nsivabalan commented on issue #4230: URL: https://github.com/apache/hudi/issues/4230#issuecomment-991830703 @BenjMaq : Is timeline server running? Can you try disabling timeline server explicitly and giving it a try in the mean time ethan assists us here. hoodie.embed.timeline.server=fa

[GitHub] [hudi] nsivabalan edited a comment on issue #4230: [SUPPORT] org.apache.hudi.exception.HoodieRemoteException: Failed to create marker file

nsivabalan edited a comment on issue #4230: URL: https://github.com/apache/hudi/issues/4230#issuecomment-991830703 @BenjMaq : Is timeline server running? Just to unblock you: Can you try disabling timeline server explicitly and giving it a try in the mean time ethan assists us here.

[GitHub] [hudi] nsivabalan commented on issue #4227: [SUPPORT] java.lang.IllegalStateException: Duplicate key Option

nsivabalan commented on issue #4227: URL: https://github.com/apache/hudi/issues/4227#issuecomment-991830847 some of your text is in non-english. but if I am not wrong, its about having duplicate entires in the map returned from getPendingRollbackInfos. Can you put up a patch with the fix

[GitHub] [hudi] nsivabalan commented on issue #4176: [SUPPORT] Getting java.lang.NoSuchFieldError: TIMESTAMP_MICROS error while hive sync using run_sync_tool.sh

nsivabalan commented on issue #4176: URL: https://github.com/apache/hudi/issues/4176#issuecomment-991831103 @codejoyan : Did the proposed solution work or are you still having issues. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] nsivabalan commented on issue #4162: [SUPPORT] First written data disappears when I append second data for schema evolution in example code.

nsivabalan commented on issue #4162: URL: https://github.com/apache/hudi/issues/4162#issuecomment-991831448 @xushiyan : with 0.10.0, we have made default compile version for spark3 as 3.2.0 right? is this going to cause any serious issue then? -- This is an automated message from the Ap

[GitHub] [hudi] hudi-bot removed a comment on pull request #4270: [HUDI-2811] Support Spark 3.2

hudi-bot removed a comment on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-991519008 ## CI report: * e68824ee98d8dd75f0b8940da24bc8a52301fdd7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4270: [HUDI-2811] Support Spark 3.2

hudi-bot commented on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-991832128 ## CI report: * e68824ee98d8dd75f0b8940da24bc8a52301fdd7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] york-yu-ctw commented on issue #4283: [SUPPORT] Data written by hudi 0.10.0 is not able be query by redshift

york-yu-ctw commented on issue #4283: URL: https://github.com/apache/hudi/issues/4283#issuecomment-991832170 @nsivabalan > Can you raise a support ticket against redshift? looks like hudi::ParsedFilename::IsValidCommitTimestamp has to be fixed when using 0.10.0. I already did, bu

[GitHub] [hudi] YannByron commented on pull request #4270: [HUDI-2811] Support Spark 3.2

YannByron commented on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-991832207 @leesf updated this pr: add explanations and optimize codes for spark3.2. -- This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [hudi] hudi-bot commented on pull request #4270: [HUDI-2811] Support Spark 3.2

hudi-bot commented on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-991832391 ## CI report: * e68824ee98d8dd75f0b8940da24bc8a52301fdd7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4270: [HUDI-2811] Support Spark 3.2

hudi-bot removed a comment on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-991832128 ## CI report: * e68824ee98d8dd75f0b8940da24bc8a52301fdd7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] yihua merged pull request #4232: [HUDI-2946] Upgrade maven plugins to be compatible with higher Java versions

yihua merged pull request #4232: URL: https://github.com/apache/hudi/pull/4232 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[hudi] branch master updated (8dd0444 -> 15444c9)

This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 8dd0444 [HUDI-2984] Implement #close for AbstractTableFileSystemView (#4285) add 15444c9 [HUDI-2946] Upgrade mav

[GitHub] [hudi] york-yu-ctw edited a comment on issue #4283: [SUPPORT] Data written by hudi 0.10.0 is not able be queried by redshift

york-yu-ctw edited a comment on issue #4283: URL: https://github.com/apache/hudi/issues/4283#issuecomment-991832170 @nsivabalan > Can you raise a support ticket against redshift? looks like hudi::ParsedFilename::IsValidCommitTimestamp has to be fixed when using 0.10.0. I already

[jira] [Commented] (HUDI-2761) IllegalArgException from timeline server when serving getLastestBaseFiles with multi-writer

[ https://issues.apache.org/jira/browse/HUDI-2761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17457810#comment-17457810 ] ZiyueGuan commented on HUDI-2761: - Close 2400 as it seems to be the same problem with this

[jira] [Commented] (HUDI-2400) Allow timeline server correctly sync when concurrent write to timeline

[ https://issues.apache.org/jira/browse/HUDI-2400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17457811#comment-17457811 ] ZiyueGuan commented on HUDI-2400: - Duplicate with https://issues.apache.org/jira/browse/HU

[jira] [Closed] (HUDI-2400) Allow timeline server correctly sync when concurrent write to timeline

[ https://issues.apache.org/jira/browse/HUDI-2400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZiyueGuan closed HUDI-2400. --- Resolution: Duplicate > Allow timeline server correctly sync when concurrent write to timeline > -

[GitHub] [hudi] yihua merged pull request #4218: [HUDI-2938] Metadata table util to get latest file slices for readers/writers

yihua merged pull request #4218: URL: https://github.com/apache/hudi/pull/4218 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[hudi] branch master updated (15444c9 -> b22c2c6)

This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 15444c9 [HUDI-2946] Upgrade maven plugins to be compatible with higher Java versions (#4232) add b22c2c6 [HUDI-2

[GitHub] [hudi] yihua merged pull request #4256: Update quick-start-guide.md - incremental query

yihua merged pull request #4256: URL: https://github.com/apache/hudi/pull/4256 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[hudi] branch asf-site updated: [MINOR] Update quick-start-guide.md - incremental query (#4256)

This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new cb461f9 [MINOR] Update quick-start-guide.md -

[GitHub] [hudi] yihua merged pull request #4220: Update quick-start-guide.md

yihua merged pull request #4220: URL: https://github.com/apache/hudi/pull/4220 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[hudi] branch asf-site updated: [MINOR] Update quick-start-guide.md (#4220)

This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 6f6f3bf [MINOR] Update quick-start-guide.md (#

[GitHub] [hudi] hudi-bot commented on pull request #4270: [HUDI-2811] Support Spark 3.2

hudi-bot commented on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-991835596 ## CI report: * e515cb1eadc04d0c5d04cf170e4af4724f5bd336 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4270: [HUDI-2811] Support Spark 3.2

hudi-bot removed a comment on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-991832391 ## CI report: * e68824ee98d8dd75f0b8940da24bc8a52301fdd7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] nsivabalan commented on issue #4086: [SUPPORT] Hudi logs error but does not fail job

nsivabalan commented on issue #4086: URL: https://github.com/apache/hudi/issues/4086#issuecomment-991836273 surprising that job succeeds despite the errors. I have not encountered this issue w/ spark-shell or deltastreamer locally as well as in EMR. but don't have exp with glue job. @um

[GitHub] [hudi] nsivabalan commented on issue #4082: [SUPPORT] How to write multiple HUDi tables simultaneously in a Spark Streaming task?

nsivabalan commented on issue #4082: URL: https://github.com/apache/hudi/issues/4082#issuecomment-991836397 may I know whats your requirement here? reading from one stream, but depending on certain conditions, write to different hudi tables? or read from multiple streams and write to diff

[GitHub] [hudi] nsivabalan commented on issue #4072: [SUPPORT]Exception in thread "main" java.io.FileNotFoundException: File does not exist: hdfs://localhost:9000/scala/table6

nsivabalan commented on issue #4072: URL: https://github.com/apache/hudi/issues/4072#issuecomment-991836698 hmmm, seems strange. if you got unblocked, can we close the ticket. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] nsivabalan edited a comment on issue #4072: [SUPPORT]Exception in thread "main" java.io.FileNotFoundException: File does not exist: hdfs://localhost:9000/scala/table6

nsivabalan edited a comment on issue #4072: URL: https://github.com/apache/hudi/issues/4072#issuecomment-991836698 hmmm, seems strange. have you tried giving a diff warehouse dir? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] xushiyan commented on issue #4162: [SUPPORT] First written data disappears when I append second data for schema evolution in example code.

xushiyan commented on issue #4162: URL: https://github.com/apache/hudi/issues/4162#issuecomment-991838374 > @xushiyan : with 0.10.0, we have made default compile version for spark3 as 3.2.0 right? is this going to cause any serious issue then? in 0.10.0 it's built against spark 3.1.2

[GitHub] [hudi] yanenze commented on issue #4227: [SUPPORT] java.lang.IllegalStateException: Duplicate key Option

yanenze commented on issue #4227: URL: https://github.com/apache/hudi/issues/4227#issuecomment-991838637 -- 原始邮件 -- 发件人: "apache/hudi"

[GitHub] [hudi] xushiyan commented on issue #4162: [SUPPORT] First written data disappears when I append second data for schema evolution in example code.

xushiyan commented on issue #4162: URL: https://github.com/apache/hudi/issues/4162#issuecomment-991838841 > @xushiyan I found a cause of this problem when I'm using sbt. [The embedded-kafka library](https://github.com/embeddedkafka/embedded-kafka) for testing kafka streaming had been cause

[GitHub] [hudi] yanenze commented on issue #4227: [SUPPORT] java.lang.IllegalStateException: Duplicate key Option

yanenze commented on issue #4227: URL: https://github.com/apache/hudi/issues/4227#issuecomment-991838894 > some of your text is in non-english. but if I am not wrong, its about having duplicate entires in the map returned from getPendingRollbackInfos. Can you put up a patch with the fix yo

[GitHub] [hudi] hudi-bot removed a comment on pull request #4067: [HUDI-2763] Metadata table records key deduplication

hudi-bot removed a comment on pull request #4067: URL: https://github.com/apache/hudi/pull/4067#issuecomment-988419193 ## CI report: * fbbbaf708aac51f283b0bab1b57c98569e2c32d6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4067: [HUDI-2763] Metadata table records key deduplication

hudi-bot commented on pull request #4067: URL: https://github.com/apache/hudi/pull/4067#issuecomment-991839180 ## CI report: * fbbbaf708aac51f283b0bab1b57c98569e2c32d6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] xushiyan edited a comment on issue #4162: [SUPPORT] First written data disappears when I append second data for schema evolution in example code.

xushiyan edited a comment on issue #4162: URL: https://github.com/apache/hudi/issues/4162#issuecomment-991838841 > @xushiyan I found a cause of this problem when I'm using sbt. [The embedded-kafka library](https://github.com/embeddedkafka/embedded-kafka) for testing kafka streaming had bee

  1   2   3   >