[GitHub] [hudi] hudi-bot commented on pull request #4984: [HUDI-3583] Fix MarkerBasedRollbackStrategy NoSuchElementException

hudi-bot commented on pull request #4984: URL: https://github.com/apache/hudi/pull/4984#issuecomment-1064867430 ## CI report: * 015f7f0e07d3f0efbd8d3a728f802fc5572a8f52 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4984: [HUDI-3583] Fix MarkerBasedRollbackStrategy NoSuchElementException

hudi-bot removed a comment on pull request #4984: URL: https://github.com/apache/hudi/pull/4984#issuecomment-1064865517 ## CI report: * 015f7f0e07d3f0efbd8d3a728f802fc5572a8f52 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4925: [HUDI-3103] Enable MultiTableDeltaStreamer to update a single sink table from multiple source tables

hudi-bot commented on pull request #4925: URL: https://github.com/apache/hudi/pull/4925#issuecomment-1064873216 ## CI report: * 018bb851445f7eabaa0bd4cc2b362f269d6fec59 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4925: [HUDI-3103] Enable MultiTableDeltaStreamer to update a single sink table from multiple source tables

hudi-bot removed a comment on pull request #4925: URL: https://github.com/apache/hudi/pull/4925#issuecomment-1064843916 ## CI report: * 018bb851445f7eabaa0bd4cc2b362f269d6fec59 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #5017: [HUDI-3606] Add `org.objenesis:objenesis` to hudi-timeline-server-bundle pom

hudi-bot removed a comment on pull request #5017: URL: https://github.com/apache/hudi/pull/5017#issuecomment-1064800761 ## CI report: * d1211dd592bcb9e3df60b80b9585d2eda9f0b8ab Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #5017: [HUDI-3606] Add `org.objenesis:objenesis` to hudi-timeline-server-bundle pom

hudi-bot commented on pull request #5017: URL: https://github.com/apache/hudi/pull/5017#issuecomment-1064873400 ## CI report: * d1211dd592bcb9e3df60b80b9585d2eda9f0b8ab Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4925: [HUDI-3103] Enable MultiTableDeltaStreamer to update a single sink table from multiple source tables

hudi-bot removed a comment on pull request #4925: URL: https://github.com/apache/hudi/pull/4925#issuecomment-1064873216 ## CI report: * 018bb851445f7eabaa0bd4cc2b362f269d6fec59 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4925: [HUDI-3103] Enable MultiTableDeltaStreamer to update a single sink table from multiple source tables

hudi-bot commented on pull request #4925: URL: https://github.com/apache/hudi/pull/4925#issuecomment-1064875199 ## CI report: * 7119319af35fb23afa97e058cd2fbfaea18292a1 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot commented on pull request #4925: [HUDI-3103] Enable MultiTableDeltaStreamer to update a single sink table from multiple source tables

hudi-bot commented on pull request #4925: URL: https://github.com/apache/hudi/pull/4925#issuecomment-1064883259 ## CI report: * 7119319af35fb23afa97e058cd2fbfaea18292a1 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #4925: [HUDI-3103] Enable MultiTableDeltaStreamer to update a single sink table from multiple source tables

hudi-bot removed a comment on pull request #4925: URL: https://github.com/apache/hudi/pull/4925#issuecomment-1064875199 ## CI report: * 7119319af35fb23afa97e058cd2fbfaea18292a1 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4489: [HUDI-3135] Fix Delete partitions with metadata table and fix show partitions in spark sql

hudi-bot commented on pull request #4489: URL: https://github.com/apache/hudi/pull/4489#issuecomment-1064900633 ## CI report: * d17343318be38b5a9b0953004700aa72f4fed689 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4489: [HUDI-3135] Fix Delete partitions with metadata table and fix show partitions in spark sql

hudi-bot removed a comment on pull request #4489: URL: https://github.com/apache/hudi/pull/4489#issuecomment-1064751722 ## CI report: * d17343318be38b5a9b0953004700aa72f4fed689 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #5018: [HUDI-3559] fix flink Bucket Index with COW table type `NoSuchElementException` cause of deduplicateRecords method in FlinkWriteHelper out of

hudi-bot commented on pull request #5018: URL: https://github.com/apache/hudi/pull/5018#issuecomment-1064901069 ## CI report: * b9e437b2c2942ba29945d1d21c7e214e350e4333 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #5018: [HUDI-3559] fix flink Bucket Index with COW table type `NoSuchElementException` cause of deduplicateRecords method in FlinkWriteHelper

hudi-bot removed a comment on pull request #5018: URL: https://github.com/apache/hudi/pull/5018#issuecomment-1064807946 ## CI report: * b9e437b2c2942ba29945d1d21c7e214e350e4333 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4489: [HUDI-3135] Fix Delete partitions with metadata table and fix show partitions in spark sql

hudi-bot commented on pull request #4489: URL: https://github.com/apache/hudi/pull/4489#issuecomment-1064902635 ## CI report: * d17343318be38b5a9b0953004700aa72f4fed689 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4489: [HUDI-3135] Fix Delete partitions with metadata table and fix show partitions in spark sql

hudi-bot removed a comment on pull request #4489: URL: https://github.com/apache/hudi/pull/4489#issuecomment-1064900633 ## CI report: * d17343318be38b5a9b0953004700aa72f4fed689 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] nloneday commented on issue #4943: [SUPPORT] NoClassDefFoundError: org/apache/hudi/org/apache/hadoop/hive/metastore/api/NoSuchObjectException

nloneday commented on issue #4943: URL: https://github.com/apache/hudi/issues/4943#issuecomment-1064910256 > > > > > > > > > @danny0405 remote the hive shade pattern in the hudi flink bundle jar ? How can I fix it now? > > You may take a reference of this pom: https://gi

[GitHub] [hudi] hudi-bot removed a comment on pull request #4971: [HUDI-3556] Re-use rollback instant for rolling back of clustering and compaction if rollback failed mid-way

hudi-bot removed a comment on pull request #4971: URL: https://github.com/apache/hudi/pull/4971#issuecomment-1064815294 ## CI report: * 74ace6ca3f717a41d54047bb44ea52fedb94e1ce Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4971: [HUDI-3556] Re-use rollback instant for rolling back of clustering and compaction if rollback failed mid-way

hudi-bot commented on pull request #4971: URL: https://github.com/apache/hudi/pull/4971#issuecomment-1064923276 ## CI report: * 7367ebfc60119b4442988ebc7350e4daac15b65f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[jira] [Created] (HUDI-3608) Support Tinyint and small int data types

Danny Chen created HUDI-3608: Summary: Support Tinyint and small int data types Key: HUDI-3608 URL: https://issues.apache.org/jira/browse/HUDI-3608 Project: Apache Hudi Issue Type: Task

[GitHub] [hudi] danny0405 commented on issue #4998: [SUPPORT] Tinyint and small int data types

danny0405 commented on issue #4998: URL: https://github.com/apache/hudi/issues/4998#issuecomment-1064924607 issue created: https://issues.apache.org/jira/browse/HUDI-3608 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[GitHub] [hudi] danny0405 closed issue #4998: [SUPPORT] Tinyint and small int data types

danny0405 closed issue #4998: URL: https://github.com/apache/hudi/issues/4998 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@

[jira] [Updated] (HUDI-3608) Support Tinyint and small int data types

[ https://issues.apache.org/jira/browse/HUDI-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-3608: - Description: Currently, the avro schema does not support tinyint and smallint datatypes, but Hudi uses the

[GitHub] [hudi] danny0405 opened a new issue #5020: [SUPPORT] The cleaning strategy breaks the reader view completeness

danny0405 opened a new issue #5020: URL: https://github.com/apache/hudi/issues/5020 Current we have some cleaning strategy such as: `num_commits`, `delta hours`, `num_versions`. Let's say user use the `num_commits` strategy. And it uses the params: - max 10 commits to archi

[GitHub] [hudi] prashantwason commented on a change in pull request #4693: [WIP][HUDI-3175][RFC-45] Implement async metadata indexing

prashantwason commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r824490905 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java ## @@ -855,6 +856,21 @@ public boolean sc

[GitHub] [hudi] hudi-bot removed a comment on pull request #4982: [HUDI-3567] Refactor HoodieCommonUtils to make code more reasonable

hudi-bot removed a comment on pull request #4982: URL: https://github.com/apache/hudi/pull/4982#issuecomment-1064820630 ## CI report: * 282ca401f8e2a93d7703f592041b854959291d41 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4982: [HUDI-3567] Refactor HoodieCommonUtils to make code more reasonable

hudi-bot commented on pull request #4982: URL: https://github.com/apache/hudi/pull/4982#issuecomment-1064953069 ## CI report: * 282ca401f8e2a93d7703f592041b854959291d41 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] prashantwason commented on a change in pull request #4693: [WIP][HUDI-3175][RFC-45] Implement async metadata indexing

prashantwason commented on a change in pull request #4693: URL: https://github.com/apache/hudi/pull/4693#discussion_r824571450 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java ## @@ -118,18 +124,18 @@ /

[GitHub] [hudi] xushiyan commented on a change in pull request #4962: [HUDI-3355] Issue with out of order commits in the timeline when ingestion writers using SparkAllowUpdateStrategy

xushiyan commented on a change in pull request #4962: URL: https://github.com/apache/hudi/pull/4962#discussion_r824584368 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java ## @@ -120,6 +121,7 @@ protected transient

[GitHub] [hudi] wangxianghu commented on pull request #4987: [HUDI-3547] Introduce MaxwellSourcePostProcessor to extract data from Maxwell json string

wangxianghu commented on pull request #4987: URL: https://github.com/apache/hudi/pull/4987#issuecomment-1064986515 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [hudi] hudi-bot removed a comment on pull request #4987: [HUDI-3547] Introduce MaxwellSourcePostProcessor to extract data from Maxwell json string

hudi-bot removed a comment on pull request #4987: URL: https://github.com/apache/hudi/pull/4987#issuecomment-1063269660 ## CI report: * 84d9028db0242b31b9fcee5cc27a361bd3c987ae UNKNOWN * 1077705483682eca8c063671fcacdf73740dacdb Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot commented on pull request #4987: [HUDI-3547] Introduce MaxwellSourcePostProcessor to extract data from Maxwell json string

hudi-bot commented on pull request #4987: URL: https://github.com/apache/hudi/pull/4987#issuecomment-1064987781 ## CI report: * 84d9028db0242b31b9fcee5cc27a361bd3c987ae UNKNOWN * 1077705483682eca8c063671fcacdf73740dacdb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot commented on pull request #4264: [HUDI-2875] Make HoodieParquetWriter Thread safe and memory executor …

hudi-bot commented on pull request #4264: URL: https://github.com/apache/hudi/pull/4264#issuecomment-1064991300 ## CI report: * 4a3662cb03d0fbf4f5041b9b27eebd03cd132783 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4264: [HUDI-2875] Make HoodieParquetWriter Thread safe and memory executor …

hudi-bot removed a comment on pull request #4264: URL: https://github.com/apache/hudi/pull/4264#issuecomment-1064838694 ## CI report: * 4a9c78781cc4efcf3f13d6f12836b6fc3e738878 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] leesf merged pull request #4926: [HUDI-3566]add thread factory in BoundedInMemoryExecutor

leesf merged pull request #4926: URL: https://github.com/apache/hudi/pull/4926 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[hudi] branch master updated (18cdad9 -> faed699)

This is an automated email from the ASF dual-hosted git repository. leesf pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 18cdad9 [HUDI-2999] [RFC-42] RFC for consistent hashing index (#4326) add faed699 [HUDI-3566] Add thread factory

[GitHub] [hudi] hudi-bot removed a comment on pull request #5019: [HUDI-3575] Use HoodieTestDataGenerator#TRIP_SCHEMA as example schema in TestSchemaPostProcessor

hudi-bot removed a comment on pull request #5019: URL: https://github.com/apache/hudi/pull/5019#issuecomment-1064842384 ## CI report: * 3b6b326bb3650689e8ad78504ccaca3df2700998 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #5019: [HUDI-3575] Use HoodieTestDataGenerator#TRIP_SCHEMA as example schema in TestSchemaPostProcessor

hudi-bot commented on pull request #5019: URL: https://github.com/apache/hudi/pull/5019#issuecomment-1065006472 ## CI report: * 3b6b326bb3650689e8ad78504ccaca3df2700998 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] wangxianghu merged pull request #5019: [HUDI-3575] Use HoodieTestDataGenerator#TRIP_SCHEMA as example schema in TestSchemaPostProcessor

wangxianghu merged pull request #5019: URL: https://github.com/apache/hudi/pull/5019 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsub

[hudi] branch master updated (faed699 -> b001803)

This is an automated email from the ASF dual-hosted git repository. wangxianghu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from faed699 [HUDI-3566] Add thread factory in BoundedInMemoryExecutor (#4926) add b001803 [HUDI-3575] Use Hood

[jira] [Closed] (HUDI-3575) Use HoodieTestDataGenerator#TRIP_SCHEMA as example schema in TestSchemaPostProcessor

[ https://issues.apache.org/jira/browse/HUDI-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-3575. -- Resolution: Fixed > Use HoodieTestDataGenerator#TRIP_SCHEMA as example schema in > TestSchemaPostProces

[GitHub] [hudi] codope commented on a change in pull request #4640: [HUDI-3225] [RFC-45] for async metadata indexing

codope commented on a change in pull request #4640: URL: https://github.com/apache/hudi/pull/4640#discussion_r824623608 ## File path: rfc/rfc-45/rfc-45.md ## @@ -0,0 +1,264 @@ + + +# RFC-45: Asynchronous Metadata Indexing + +## Proposers + +- @codope +- @manojpec + +## Approver

[GitHub] [hudi] hudi-bot commented on pull request #4872: [HUDI-3475] Support run compaction / clustering job in Service

hudi-bot commented on pull request #4872: URL: https://github.com/apache/hudi/pull/4872#issuecomment-1065023815 ## CI report: * c662e400cd71c1dbba9b4f37512ca5e748736f03 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4872: [HUDI-3475] Support run compaction / clustering job in Service

hudi-bot removed a comment on pull request #4872: URL: https://github.com/apache/hudi/pull/4872#issuecomment-1064853531 ## CI report: * 0fd561ae050f39c022862eae351c73b323a61e05 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] xushiyan commented on a change in pull request #4175: [HUDI-2883] Refactor hive sync tool / config to use reflection and standardize configs

xushiyan commented on a change in pull request #4175: URL: https://github.com/apache/hudi/pull/4175#discussion_r824606178 ## File path: hudi-sync/hudi-hive-sync/src/test/java/org/apache/hudi/hive/testutils/HiveTestUtil.java ## @@ -89,16 +89,21 @@ @SuppressWarnings("SameParame

[GitHub] [hudi] hudi-bot removed a comment on pull request #5013: [HUDI-3593] Restore TypedProperties and flush checksum in table config

hudi-bot removed a comment on pull request #5013: URL: https://github.com/apache/hudi/pull/5013#issuecomment-1064845719 ## CI report: * a2e2b2ecd3ffe2974fac5e6472c2ab273f4d13c4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #5013: [HUDI-3593] Restore TypedProperties and flush checksum in table config

hudi-bot commented on pull request #5013: URL: https://github.com/apache/hudi/pull/5013#issuecomment-1065024087 ## CI report: * f50fc2686b0c3b7f17c741ca99db9629aafc6b66 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] huberylee commented on pull request #4982: [HUDI-3567] Refactor HoodieCommonUtils to make code more reasonable

huberylee commented on pull request #4982: URL: https://github.com/apache/hudi/pull/4982#issuecomment-1065024812 > @huberylee thanks for cleaning things up! @alexeykudinkin All comments have been addressed. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] boneanxs commented on a change in pull request #5013: [HUDI-3593] Restore TypedProperties and flush checksum in table config

boneanxs commented on a change in pull request #5013: URL: https://github.com/apache/hudi/pull/5013#discussion_r824633524 ## File path: hudi-common/src/main/java/org/apache/hudi/common/config/OrderedProperties.java ## @@ -0,0 +1,150 @@ +/* + * Licensed to the Apache Software F

[jira] [Commented] (HUDI-3607) Support backend switch in HoodieFlinkStreamer

[ https://issues.apache.org/jira/browse/HUDI-3607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17504872#comment-17504872 ] Xianghu Wang commented on HUDI-3607: hi [~liufangqi]  The reasion you can't assign it

[jira] [Comment Edited] (HUDI-3607) Support backend switch in HoodieFlinkStreamer

[ https://issues.apache.org/jira/browse/HUDI-3607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17504872#comment-17504872 ] Xianghu Wang edited comment on HUDI-3607 at 3/11/22, 11:39 AM: -

[GitHub] [hudi] codope commented on a change in pull request #4640: [HUDI-3225] [RFC-45] for async metadata indexing

codope commented on a change in pull request #4640: URL: https://github.com/apache/hudi/pull/4640#discussion_r824640318 ## File path: rfc/rfc-45/rfc-45.md ## @@ -0,0 +1,264 @@ + + +# RFC-45: Asynchronous Metadata Indexing + +## Proposers + +- @codope +- @manojpec + +## Approver

[GitHub] [hudi] vinothchandar commented on pull request #4012: [HUDI-2777] Data import performance deteriorates because multiple Spark jobs are started when data is written to disks.

vinothchandar commented on pull request #4012: URL: https://github.com/apache/hudi/pull/4012#issuecomment-1065043222 @xushiyan @nsivabalan can we get to the bottom of this issue? and summarize what we find. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [hudi] codope commented on a change in pull request #4640: [HUDI-3225] [RFC-45] for async metadata indexing

codope commented on a change in pull request #4640: URL: https://github.com/apache/hudi/pull/4640#discussion_r824647337 ## File path: rfc/rfc-45/rfc-45.md ## @@ -0,0 +1,229 @@ + + +# RFC-45: Asynchronous Metadata Indexing + +## Proposers + +- @codope +- @manojpec + +## Approver

[GitHub] [hudi] vinothchandar commented on a change in pull request #4264: [HUDI-2875] Make HoodieParquetWriter Thread safe and memory executor …

vinothchandar commented on a change in pull request #4264: URL: https://github.com/apache/hudi/pull/4264#discussion_r824648413 ## File path: hudi-common/src/main/java/org/apache/hudi/common/util/queue/BoundedInMemoryExecutor.java ## @@ -47,7 +48,7 @@ public class BoundedInMem

[GitHub] [hudi] vinothchandar commented on pull request #4880: [HUDI-2752] The MOR DELETE block breaks the event time sequence of CDC

vinothchandar commented on pull request #4880: URL: https://github.com/apache/hudi/pull/4880#issuecomment-1065050025 Will review again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [hudi] hudi-bot removed a comment on pull request #4984: [HUDI-3583] Fix MarkerBasedRollbackStrategy NoSuchElementException

hudi-bot removed a comment on pull request #4984: URL: https://github.com/apache/hudi/pull/4984#issuecomment-1064867430 ## CI report: * 015f7f0e07d3f0efbd8d3a728f802fc5572a8f52 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4984: [HUDI-3583] Fix MarkerBasedRollbackStrategy NoSuchElementException

hudi-bot commented on pull request #4984: URL: https://github.com/apache/hudi/pull/4984#issuecomment-1065050643 ## CI report: * e30a63cc90f3afbea7ee36c37283f2f21ea7998f UNKNOWN * c0a0e141561d1d75150aab046090e1ccd1c9e2c2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4640: [HUDI-3225] [RFC-45] for async metadata indexing

hudi-bot removed a comment on pull request #4640: URL: https://github.com/apache/hudi/pull/4640#issuecomment-1063194983 ## CI report: * 7afccec9740814a4bf7f8a3d8a6125c223829d27 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4640: [HUDI-3225] [RFC-45] for async metadata indexing

hudi-bot commented on pull request #4640: URL: https://github.com/apache/hudi/pull/4640#issuecomment-1065054306 ## CI report: * 7afccec9740814a4bf7f8a3d8a6125c223829d27 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] codope commented on a change in pull request #4640: [HUDI-3225] [RFC-45] for async metadata indexing

codope commented on a change in pull request #4640: URL: https://github.com/apache/hudi/pull/4640#discussion_r824656295 ## File path: rfc/rfc-45/rfc-45.md ## @@ -0,0 +1,229 @@ + + +# RFC-45: Asynchronous Metadata Indexing + +## Proposers + +- @codope +- @manojpec + +## Approver

[GitHub] [hudi] hudi-bot commented on pull request #4640: [HUDI-3225] [RFC-45] for async metadata indexing

hudi-bot commented on pull request #4640: URL: https://github.com/apache/hudi/pull/4640#issuecomment-1065056424 ## CI report: * 7afccec9740814a4bf7f8a3d8a6125c223829d27 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4640: [HUDI-3225] [RFC-45] for async metadata indexing

hudi-bot removed a comment on pull request #4640: URL: https://github.com/apache/hudi/pull/4640#issuecomment-1065054306 ## CI report: * 7afccec9740814a4bf7f8a3d8a6125c223829d27 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[jira] [Updated] (HUDI-2871) Decouple metrics dependencies from hudi-client-common

[ https://issues.apache.org/jira/browse/HUDI-2871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2871: - Reviewers: Rajesh Mahindra, sivabalan narayanan (was: Raymond Xu, sivabalan narayanan) > Decouple metric

[jira] [Updated] (HUDI-3435) Do not throw exception when instant to rollback does not exist in metadata table active timeline

[ https://issues.apache.org/jira/browse/HUDI-3435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3435: - Sprint: (was: Hudi-Sprint-Mar-07) > Do not throw exception when instant to rollback does not exist in me

[jira] [Updated] (HUDI-3435) Do not throw exception when instant to rollback does not exist in metadata table active timeline

[ https://issues.apache.org/jira/browse/HUDI-3435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3435: - Sprint: Cont' improve - 2022/03/7 > Do not throw exception when instant to rollback does not exist in meta

[jira] [Updated] (HUDI-2777) Data import performance deteriorates because multiple Spark jobs are started when data is written to disks.

[ https://issues.apache.org/jira/browse/HUDI-2777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2777: - Sprint: Cont' improve - 2022/03/7 > Data import performance deteriorates because multiple Spark jobs are s

[jira] [Updated] (HUDI-2777) Data import performance deteriorates because multiple Spark jobs are started when data is written to disks.

[ https://issues.apache.org/jira/browse/HUDI-2777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2777: - Labels: pull-request-available (was: pull-request-available query-eng) > Data import performance deterior

[jira] [Updated] (HUDI-2777) Data import performance deteriorates because multiple Spark jobs are started when data is written to disks.

[ https://issues.apache.org/jira/browse/HUDI-2777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2777: - Labels: performance pull-request-available (was: pull-request-available) > Data import performance deteri

[jira] [Updated] (HUDI-2777) Data import performance deteriorates because multiple Spark jobs are started when data is written to disks.

[ https://issues.apache.org/jira/browse/HUDI-2777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2777: - Labels: pull-request-available query-eng (was: hudi-on-call pull-request-available query-eng sev:high) >

[GitHub] [hudi] hudi-bot removed a comment on pull request #4925: [HUDI-3103] Enable MultiTableDeltaStreamer to update a single sink table from multiple source tables

hudi-bot removed a comment on pull request #4925: URL: https://github.com/apache/hudi/pull/4925#issuecomment-1064883259 ## CI report: * 7119319af35fb23afa97e058cd2fbfaea18292a1 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4925: [HUDI-3103] Enable MultiTableDeltaStreamer to update a single sink table from multiple source tables

hudi-bot commented on pull request #4925: URL: https://github.com/apache/hudi/pull/4925#issuecomment-1065073245 ## CI report: * 82acc7daea301fa4e373c1ff2570a60ca1da6ce3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] maddy2u commented on issue #4990: [SUPPORT] PySpark Examples for DeltaStreamer using Kinesis

maddy2u commented on issue #4990: URL: https://github.com/apache/hudi/issues/4990#issuecomment-1065076167 Hi - Any update on this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] maddy2u commented on issue #4202: [SUPPORT] Support Apache Spark 3.2

maddy2u commented on issue #4202: URL: https://github.com/apache/hudi/issues/4202#issuecomment-1065076555 It works. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] hudi-bot commented on pull request #4489: [HUDI-3135] Fix Delete partitions with metadata table and fix show partitions in spark sql

hudi-bot commented on pull request #4489: URL: https://github.com/apache/hudi/pull/4489#issuecomment-1065080414 ## CI report: * 44942ace20195bb284b5ce7e792462865255f2e0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4489: [HUDI-3135] Fix Delete partitions with metadata table and fix show partitions in spark sql

hudi-bot removed a comment on pull request #4489: URL: https://github.com/apache/hudi/pull/4489#issuecomment-1064902635 ## CI report: * d17343318be38b5a9b0953004700aa72f4fed689 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] xushiyan commented on a change in pull request #4877: [HUDI-3457][Stacked on 4818] Refactored Spark DataSource Relations to avoid code duplication

xushiyan commented on a change in pull request #4877: URL: https://github.com/apache/hudi/pull/4877#discussion_r824682968 ## File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/BaseFileOnlyRelation.scala ## @@ -0,0 +1,94 @@ +/* + * Licensed to the

[GitHub] [hudi] hudi-bot commented on pull request #4987: [HUDI-3547] Introduce MaxwellSourcePostProcessor to extract data from Maxwell json string

hudi-bot commented on pull request #4987: URL: https://github.com/apache/hudi/pull/4987#issuecomment-1065096681 ## CI report: * 84d9028db0242b31b9fcee5cc27a361bd3c987ae UNKNOWN * 1077705483682eca8c063671fcacdf73740dacdb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4987: [HUDI-3547] Introduce MaxwellSourcePostProcessor to extract data from Maxwell json string

hudi-bot removed a comment on pull request #4987: URL: https://github.com/apache/hudi/pull/4987#issuecomment-1064987781 ## CI report: * 84d9028db0242b31b9fcee5cc27a361bd3c987ae UNKNOWN * 1077705483682eca8c063671fcacdf73740dacdb Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] danny0405 commented on issue #4943: [SUPPORT] NoClassDefFoundError: org/apache/hudi/org/apache/hadoop/hive/metastore/api/NoSuchObjectException

danny0405 commented on issue #4943: URL: https://github.com/apache/hudi/issues/4943#issuecomment-1065103698 Yeah, that is a solution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot removed a comment on pull request #4693: [WIP][HUDI-3175][RFC-45] Implement async metadata indexing

hudi-bot removed a comment on pull request #4693: URL: https://github.com/apache/hudi/pull/4693#issuecomment-1033224161 ## CI report: * 7920cb15d99cd92ea2a3e6bd515249eb63040772 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4693: [WIP][HUDI-3175][RFC-45] Implement async metadata indexing

hudi-bot commented on pull request #4693: URL: https://github.com/apache/hudi/pull/4693#issuecomment-1065109086 ## CI report: * 7920cb15d99cd92ea2a3e6bd515249eb63040772 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[jira] [Commented] (HUDI-3058) SqlQueryEqualityPreCommitValidator errors with java.util.ConcurrentModificationException

[ https://issues.apache.org/jira/browse/HUDI-3058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17504918#comment-17504918 ] zhangyingjie commented on HUDI-3058: When I run two spark jobs to write to the same hu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4693: [WIP][HUDI-3175][RFC-45] Implement async metadata indexing

hudi-bot removed a comment on pull request #4693: URL: https://github.com/apache/hudi/pull/4693#issuecomment-1065109086 ## CI report: * 7920cb15d99cd92ea2a3e6bd515249eb63040772 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4693: [WIP][HUDI-3175][RFC-45] Implement async metadata indexing

hudi-bot commented on pull request #4693: URL: https://github.com/apache/hudi/pull/4693#issuecomment-1065111216 ## CI report: * 7920cb15d99cd92ea2a3e6bd515249eb63040772 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] codope commented on pull request #4693: [WIP][HUDI-3175][RFC-45] Implement async metadata indexing

codope commented on pull request #4693: URL: https://github.com/apache/hudi/pull/4693#issuecomment-1065111787 > Good progress on this one. Getting close to being complete. Thanks @prashantwason for reviewing. I'll address your comments soon. -- This is an automated message from the

[jira] [Commented] (HUDI-3593) AsyncClustering failed because of ConcurrentModificationException

[ https://issues.apache.org/jira/browse/HUDI-3593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17504921#comment-17504921 ] Sagar Sumit commented on HUDI-3593: --- [~shibei] [~Bone An] Could try out [https://github.

[jira] [Commented] (HUDI-3593) AsyncClustering failed because of ConcurrentModificationException

[ https://issues.apache.org/jira/browse/HUDI-3593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17504923#comment-17504923 ] Sagar Sumit commented on HUDI-3593: --- Also, I think changing bulk insert api is a signifi

[jira] [Commented] (HUDI-3607) Support backend switch in HoodieFlinkStreamer

[ https://issues.apache.org/jira/browse/HUDI-3607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17504933#comment-17504933 ] 刘方奇 commented on HUDI-3607: --- THX for your reply, I will try your link. > Support backend switch

[GitHub] [hudi] hudi-bot commented on pull request #4640: [HUDI-3225] [RFC-45] for async metadata indexing

hudi-bot commented on pull request #4640: URL: https://github.com/apache/hudi/pull/4640#issuecomment-1065151944 ## CI report: * 62db921e601f9c81c8bd9bb53df771aec6e2de6e Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4640: [HUDI-3225] [RFC-45] for async metadata indexing

hudi-bot removed a comment on pull request #4640: URL: https://github.com/apache/hudi/pull/4640#issuecomment-1065056424 ## CI report: * 7afccec9740814a4bf7f8a3d8a6125c223829d27 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] nsivabalan commented on a change in pull request #4640: [HUDI-3225] [RFC-45] for async metadata indexing

nsivabalan commented on a change in pull request #4640: URL: https://github.com/apache/hudi/pull/4640#discussion_r824747770 ## File path: rfc/rfc-45/rfc-45.md ## @@ -0,0 +1,281 @@ + + +# RFC-45: Asynchronous Metadata Indexing + +## Proposers + +- @codope +- @manojpec + +## Appr

[GitHub] [hudi] nsivabalan commented on a change in pull request #5013: [HUDI-3593] Restore TypedProperties and flush checksum in table config

nsivabalan commented on a change in pull request #5013: URL: https://github.com/apache/hudi/pull/5013#discussion_r824761454 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java ## @@ -233,6 +235,23 @@ public HoodieTableConfig(FileSystem f

[GitHub] [hudi] maddy2u opened a new issue #5021: [FEATURE REQUEST] Support for Presto/Athena to make updates to Hudi Tables

maddy2u opened a new issue #5021: URL: https://github.com/apache/hudi/issues/5021 Hi, Within AWS Tech Stack, we seem to have support for Athena to make updates to Apache Iceberg tables as seen here (https://docs.aws.amazon.com/athena/latest/ug/querying-iceberg.html). It woul

[GitHub] [hudi] nsivabalan commented on a change in pull request #5013: [HUDI-3593] Restore TypedProperties and flush checksum in table config

nsivabalan commented on a change in pull request #5013: URL: https://github.com/apache/hudi/pull/5013#discussion_r824766705 ## File path: hudi-common/src/main/java/org/apache/hudi/common/config/OrderedProperties.java ## @@ -0,0 +1,150 @@ +/* + * Licensed to the Apache Software

[jira] [Created] (HUDI-3609) Create scala version specific artifacts for hudi-spark-client

sivabalan narayanan created HUDI-3609: - Summary: Create scala version specific artifacts for hudi-spark-client Key: HUDI-3609 URL: https://issues.apache.org/jira/browse/HUDI-3609 Project: Apache H

[jira] [Updated] (HUDI-3609) Create scala version specific artifacts for hudi-spark-client

[ https://issues.apache.org/jira/browse/HUDI-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3609: -- Fix Version/s: 0.11.0 > Create scala version specific artifacts for hudi-spark-client >

[GitHub] [hudi] xushiyan commented on a change in pull request #4877: [HUDI-3457][Stacked on 4818] Refactored Spark DataSource Relations to avoid code duplication

xushiyan commented on a change in pull request #4877: URL: https://github.com/apache/hudi/pull/4877#discussion_r824767545 ## File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieDataSourceHelper.scala ## @@ -65,20 +65,6 @@ object HoodieDataSo

[jira] [Updated] (HUDI-2606) Ensure query engines not access MDT if disabled

[ https://issues.apache.org/jira/browse/HUDI-2606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2606: - Story Points: 0.5 (was: 0) > Ensure query engines not access MDT if disabled > --

[jira] [Updated] (HUDI-3535) Integrate with LinkedIn DataHub

[ https://issues.apache.org/jira/browse/HUDI-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3535: - Fix Version/s: 0.11.0 > Integrate with LinkedIn DataHub > --- > >

  1   2   3   4   >