[GitHub] [hudi] hudi-bot commented on pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

hudi-bot commented on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1004597630 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * 1a236d6d41630338eed7eeec90d2a28e738ed2a1 Azur

[GitHub] [hudi] hudi-bot removed a comment on pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

hudi-bot removed a comment on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-155988 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * 1a236d6d41630338eed7eeec90d2a28e738ed

[GitHub] [hudi] zhangyue19921010 commented on a change in pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

zhangyue19921010 commented on a change in pull request #4078: URL: https://github.com/apache/hudi/pull/4078#discussion_r777887971 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTimelineArchiveLog.java ## @@ -106,6 +116,23 @@ private Writ

[GitHub] [hudi] hudi-bot removed a comment on pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

hudi-bot removed a comment on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1004597630 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * 1a236d6d41630338eed7eeec90d2a28e738ed

[GitHub] [hudi] hudi-bot commented on pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

hudi-bot commented on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1004599306 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * 1a236d6d41630338eed7eeec90d2a28e738ed2a1 Azur

[GitHub] [hudi] zhangyue19921010 commented on a change in pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

zhangyue19921010 commented on a change in pull request #4078: URL: https://github.com/apache/hudi/pull/4078#discussion_r777892769 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTimelineArchiveLog.java ## @@ -134,12 +161,199 @@ public boo

[GitHub] [hudi] hudi-bot removed a comment on pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

hudi-bot removed a comment on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1004599306 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * 1a236d6d41630338eed7eeec90d2a28e738ed

[GitHub] [hudi] hudi-bot commented on pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

hudi-bot commented on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1004606006 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * 1a236d6d41630338eed7eeec90d2a28e738ed2a1 Azur

[GitHub] [hudi] zhangyue19921010 commented on a change in pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

zhangyue19921010 commented on a change in pull request #4078: URL: https://github.com/apache/hudi/pull/4078#discussion_r777895112 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTimelineArchiveLog.java ## @@ -134,12 +161,199 @@ public boo

[GitHub] [hudi] zhangyue19921010 commented on a change in pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

zhangyue19921010 commented on a change in pull request #4078: URL: https://github.com/apache/hudi/pull/4078#discussion_r777895319 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTimelineArchiveLog.java ## @@ -134,12 +161,199 @@ public boo

[GitHub] [hudi] zhangyue19921010 commented on a change in pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

zhangyue19921010 commented on a change in pull request #4078: URL: https://github.com/apache/hudi/pull/4078#discussion_r777895427 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTimelineArchiveLog.java ## @@ -134,12 +161,199 @@ public boo

[GitHub] [hudi] zhangyue19921010 commented on a change in pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

zhangyue19921010 commented on a change in pull request #4078: URL: https://github.com/apache/hudi/pull/4078#discussion_r777896842 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieArchivedTimeline.java ## @@ -218,7 +219,7 @@ private HoodieInst

[GitHub] [hudi] zhangyue19921010 commented on a change in pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

zhangyue19921010 commented on a change in pull request #4078: URL: https://github.com/apache/hudi/pull/4078#discussion_r777898257 ## File path: hudi-common/src/main/avro/HoodieMergeArchiveFilePlan.avsc ## @@ -0,0 +1,37 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [hudi] zhangyue19921010 commented on a change in pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

zhangyue19921010 commented on a change in pull request #4078: URL: https://github.com/apache/hudi/pull/4078#discussion_r777892769 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTimelineArchiveLog.java ## @@ -134,12 +161,199 @@ public boo

[GitHub] [hudi] hudi-bot commented on pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

hudi-bot commented on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1004611017 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * 1a236d6d41630338eed7eeec90d2a28e738ed2a1 Azur

[GitHub] [hudi] hudi-bot removed a comment on pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

hudi-bot removed a comment on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1004606006 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * 1a236d6d41630338eed7eeec90d2a28e738ed

[GitHub] [hudi] zhangyue19921010 commented on a change in pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

zhangyue19921010 commented on a change in pull request #4078: URL: https://github.com/apache/hudi/pull/4078#discussion_r777901212 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTimelineArchiveLog.java ## @@ -134,12 +161,199 @@ public boo

[GitHub] [hudi] hudi-bot commented on pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

hudi-bot commented on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1004614497 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * 1a236d6d41630338eed7eeec90d2a28e738ed2a1 Azur

[GitHub] [hudi] hudi-bot removed a comment on pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

hudi-bot removed a comment on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1004611017 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * 1a236d6d41630338eed7eeec90d2a28e738ed

[GitHub] [hudi] zhangyue19921010 commented on a change in pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

zhangyue19921010 commented on a change in pull request #4078: URL: https://github.com/apache/hudi/pull/4078#discussion_r777896842 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieArchivedTimeline.java ## @@ -218,7 +219,7 @@ private HoodieInst

[GitHub] [hudi] zhangyue19921010 commented on a change in pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

zhangyue19921010 commented on a change in pull request #4078: URL: https://github.com/apache/hudi/pull/4078#discussion_r777896842 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieArchivedTimeline.java ## @@ -218,7 +219,7 @@ private HoodieInst

[GitHub] [hudi] zhangyue19921010 commented on a change in pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

zhangyue19921010 commented on a change in pull request #4078: URL: https://github.com/apache/hudi/pull/4078#discussion_r777896842 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieArchivedTimeline.java ## @@ -218,7 +219,7 @@ private HoodieInst

[GitHub] [hudi] zhangyue19921010 commented on a change in pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

zhangyue19921010 commented on a change in pull request #4078: URL: https://github.com/apache/hudi/pull/4078#discussion_r777924466 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieArchivedTimeline.java ## @@ -218,7 +219,7 @@ private HoodieInst

[jira] [Created] (HUDI-3161) Add Call Produce Command for spark sql

Forward Xu created HUDI-3161: Summary: Add Call Produce Command for spark sql Key: HUDI-3161 URL: https://issues.apache.org/jira/browse/HUDI-3161 Project: Apache Hudi Issue Type: New Feature

[GitHub] [hudi] manojpec opened a new pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when records by keys

manojpec opened a new pull request #4505: URL: https://github.com/apache/hudi/pull/4505 ## What is the purpose of the pull request - HoodieMetadataMergedLogRecordReader#getRecordsByKeys() and its parent class methods are not thread safe. When multiple queries come in for gettting

[jira] [Updated] (HUDI-3141) Metadata table getAllFilesInPartition() crashes with NullPointerException

[ https://issues.apache.org/jira/browse/HUDI-3141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HUDI-3141: - Reviewers: sivabalan narayanan > Metadata table getAllFilesInPartition() crashes with Null

[jira] [Updated] (HUDI-3141) Metadata table getAllFilesInPartition() crashes with NullPointerException

[ https://issues.apache.org/jira/browse/HUDI-3141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3141: - Labels: pull-request-available (was: ) > Metadata table getAllFilesInPartition() crashes with Nul

[GitHub] [hudi] hudi-bot commented on pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys

hudi-bot commented on pull request #4505: URL: https://github.com/apache/hudi/pull/4505#issuecomment-1004644941 ## CI report: * 15c4b754f2f0a5d085e3b372abbfbba4fa24c69d UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot commented on pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys

hudi-bot commented on pull request #4505: URL: https://github.com/apache/hudi/pull/4505#issuecomment-1004646987 ## CI report: * 15c4b754f2f0a5d085e3b372abbfbba4fa24c69d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys

hudi-bot removed a comment on pull request #4505: URL: https://github.com/apache/hudi/pull/4505#issuecomment-1004644941 ## CI report: * 15c4b754f2f0a5d085e3b372abbfbba4fa24c69d UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] YuweiXiao commented on pull request #4480: [HUDI-3123] consistent hashing index: basic write path (upsert/insert)

YuweiXiao commented on pull request #4480: URL: https://github.com/apache/hudi/pull/4480#issuecomment-1004647179 cc @minihippo -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [hudi] hudi-bot removed a comment on pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

hudi-bot removed a comment on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1004614497 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * 1a236d6d41630338eed7eeec90d2a28e738ed

[GitHub] [hudi] hudi-bot commented on pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

hudi-bot commented on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1004650667 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * c36aac530d3350857fb01df858d0f26c123e5766 UNKN

[GitHub] [hudi] zhangyue19921010 commented on pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

zhangyue19921010 commented on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1004657829 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [hudi] hudi-bot commented on pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

hudi-bot commented on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1004658740 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * c36aac530d3350857fb01df858d0f26c123e5766 UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

hudi-bot removed a comment on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1004650667 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * c36aac530d3350857fb01df858d0f26c123e5

[GitHub] [hudi] xushiyan commented on issue #4474: [SUPPORT] Should we shade all aws dependencies to avoid class conflicts?

xushiyan commented on issue #4474: URL: https://github.com/apache/hudi/issues/4474#issuecomment-1004664907 @boneanxs +1 for shading. We already do that for other dependencies (global search `shade-unbundle`). Feel free to file a JIRA and a patch. thanks! @umehrot2 @zhedoubushishi please ch

[GitHub] [hudi] BenjMaq commented on issue #4154: [SUPPORT] INSERT OVERWRITE operation does not work when using Spark SQL

BenjMaq commented on issue #4154: URL: https://github.com/apache/hudi/issues/4154#issuecomment-1004675484 @codope Hive is automatically synced. It works correctly for inserts, updates, deletes. I followed [these steps](https://hudi.apache.org/docs/query_engine_setup#hive) initially for Pr

[GitHub] [hudi] BenjMaq edited a comment on issue #4154: [SUPPORT] INSERT OVERWRITE operation does not work when using Spark SQL

BenjMaq edited a comment on issue #4154: URL: https://github.com/apache/hudi/issues/4154#issuecomment-1004675484 @codope Hive is automatically synced. It works correctly for inserts, updates, deletes. I followed [these steps](https://hudi.apache.org/docs/query_engine_setup#hive) initially

[GitHub] [hudi] BenjMaq edited a comment on issue #4154: [SUPPORT] INSERT OVERWRITE operation does not work when using Spark SQL

BenjMaq edited a comment on issue #4154: URL: https://github.com/apache/hudi/issues/4154#issuecomment-1004675484 @codope Hive is automatically synced. It works correctly for inserts, updates, deletes. I followed [these steps](https://hudi.apache.org/docs/query_engine_setup#hive) initially

[GitHub] [hudi] xiarixiaoyao commented on pull request #4405: HUDI-3068 Fixing sync all partitions

xiarixiaoyao commented on pull request #4405: URL: https://github.com/apache/hudi/pull/4405#issuecomment-1004680987 @nsivabalan According to the current logic, this problem is difficult to occur, because we determine whether the current partition needs alter by comparing whether the pa

[GitHub] [hudi] xiarixiaoyao edited a comment on pull request #4405: HUDI-3068 Fixing sync all partitions

xiarixiaoyao edited a comment on pull request #4405: URL: https://github.com/apache/hudi/pull/4405#issuecomment-1004680987 @nsivabalan According to the current logic, this problem is difficult to occur, because we determine whether the current partition needs alter by comparing whether

[GitHub] [hudi] xiarixiaoyao edited a comment on pull request #4405: HUDI-3068 Fixing sync all partitions

xiarixiaoyao edited a comment on pull request #4405: URL: https://github.com/apache/hudi/pull/4405#issuecomment-1004680987 @nsivabalan According to the current logic, this problem is difficult to occur, because we determine whether the current partition needs alter by comparing whether

[GitHub] [hudi] hudi-bot commented on pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys

hudi-bot commented on pull request #4505: URL: https://github.com/apache/hudi/pull/4505#issuecomment-1004684486 ## CI report: * 15c4b754f2f0a5d085e3b372abbfbba4fa24c69d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys

hudi-bot removed a comment on pull request #4505: URL: https://github.com/apache/hudi/pull/4505#issuecomment-1004646987 ## CI report: * 15c4b754f2f0a5d085e3b372abbfbba4fa24c69d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

hudi-bot removed a comment on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1004658740 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * c36aac530d3350857fb01df858d0f26c123e5

[GitHub] [hudi] hudi-bot commented on pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

hudi-bot commented on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1004700674 ## CI report: * 8f8ae385baf21dacd4b9fedd3670133160001dc0 UNKNOWN * 019e161bb908731244e13cdf36d12781956f0114 UNKNOWN * c36aac530d3350857fb01df858d0f26c123e5766 UNKN

[jira] [Created] (HUDI-3162) Shade AWS dependencies for bundled packages

Hui An created HUDI-3162: Summary: Shade AWS dependencies for bundled packages Key: HUDI-3162 URL: https://issues.apache.org/jira/browse/HUDI-3162 Project: Apache Hudi Issue Type: Improvement

[GitHub] [hudi] codope commented on issue #4154: [SUPPORT] INSERT OVERWRITE operation does not work when using Spark SQL

codope commented on issue #4154: URL: https://github.com/apache/hudi/issues/4154#issuecomment-1004760934 @BenjMaq That's right. If you did that hive/presto setup once it's enough. We don't need to sync every commit. As for the difference between commit and replacecommit, that is expected f

[GitHub] [hudi] boneanxs commented on issue #4474: [SUPPORT] Should we shade all aws dependencies to avoid class conflicts?

boneanxs commented on issue #4474: URL: https://github.com/apache/hudi/issues/4474#issuecomment-1004795295 After some investigation about this, I'm curious why flink remove amazonaws packages by this JIRA(it doesn't have too much description): [HUDI-2803](https://issues.apache.org/jira/bro

[GitHub] [hudi] boneanxs edited a comment on issue #4474: [SUPPORT] Should we shade all aws dependencies to avoid class conflicts?

boneanxs edited a comment on issue #4474: URL: https://github.com/apache/hudi/issues/4474#issuecomment-1004795295 After some investigation about this, I'm curious why flink remove amazonaws packages by this JIRA(it doesn't have too much description): [HUDI-2803](https://issues.apache.org/j

[GitHub] [hudi] boneanxs edited a comment on issue #4474: [SUPPORT] Should we shade all aws dependencies to avoid class conflicts?

boneanxs edited a comment on issue #4474: URL: https://github.com/apache/hudi/issues/4474#issuecomment-1004795295 After some investigation about this, I'm curious why flink remove amazonaws packages by this JIRA(it doesn't have too much description): [HUDI-2803](https://issues.apache.org/j

[GitHub] [hudi] BenjMaq commented on issue #4154: [SUPPORT] INSERT OVERWRITE operation does not work when using Spark SQL

BenjMaq commented on issue #4154: URL: https://github.com/apache/hudi/issues/4154#issuecomment-1004806239 I'm facing that issues using Hudi `0.9.0` and `0.10.0`. I'll try with the latest master. Thank you for following up on this! -- This is an automated message from the Apache Git Se

[GitHub] [hudi] codope opened a new pull request #4118: [HUDI-2774] Merge duplicate instants while fetching pending clustering plans

codope opened a new pull request #4118: URL: https://github.com/apache/hudi/pull/4118 ## What is the purpose of the pull request With fast ingestion through deltastreamer (no min sync interval) and frequent async clustering enabled, it could be possible that two clustering plans cre

[jira] [Updated] (HUDI-3163) Validate/certify hudi against diff spark 3 versions

[ https://issues.apache.org/jira/browse/HUDI-3163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3163: -- Labels: user-support-issues (was: ) > Validate/certify hudi against diff spark 3 versio

[jira] [Created] (HUDI-3163) Validate/certify hudi against diff spark 3 versions

sivabalan narayanan created HUDI-3163: - Summary: Validate/certify hudi against diff spark 3 versions Key: HUDI-3163 URL: https://issues.apache.org/jira/browse/HUDI-3163 Project: Apache Hudi

[jira] [Updated] (HUDI-3163) Validate/certify hudi against diff spark 3 versions

[ https://issues.apache.org/jira/browse/HUDI-3163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3163: -- Fix Version/s: 0.11.0 0.10.1 > Validate/certify hudi against diff spa

[GitHub] [hudi] codope commented on pull request #4118: [HUDI-2774] Merge duplicate instants while fetching pending clustering plans

codope commented on pull request #4118: URL: https://github.com/apache/hudi/pull/4118#issuecomment-1004824900 @nsivabalan Reopened this PR and handled the duplicate instants as we discussed. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [hudi] hudi-bot commented on pull request #4118: [HUDI-2774] Merge duplicate instants while fetching pending clustering plans

hudi-bot commented on pull request #4118: URL: https://github.com/apache/hudi/pull/4118#issuecomment-1004825337 ## CI report: * 3348904bd9d9a0c4469d3bf3c2e2cc47e995ea06 UNKNOWN * 971e52c6cc761bb9b68da6cfa3ba0935db195266 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4118: [HUDI-2774] Merge duplicate instants while fetching pending clustering plans

hudi-bot removed a comment on pull request #4118: URL: https://github.com/apache/hudi/pull/4118#issuecomment-979052450 ## CI report: * 3348904bd9d9a0c4469d3bf3c2e2cc47e995ea06 UNKNOWN * 971e52c6cc761bb9b68da6cfa3ba0935db195266 Azure: [FAILURE](https://dev.azure.com/apache-hudi

[jira] [Created] (HUDI-3164) CTAS fails w/ UnsupportedOperationException when trying to modify immutable map in DataSourceUtils.mayBeOverwriteParquetWriteLegacyFormatProp

sivabalan narayanan created HUDI-3164: - Summary: CTAS fails w/ UnsupportedOperationException when trying to modify immutable map in DataSourceUtils.mayBeOverwriteParquetWriteLegacyFormatProp Key: HUDI-3164 UR

[jira] [Updated] (HUDI-3164) CTAS fails w/ UnsupportedOperationException when trying to modify immutable map in DataSourceUtils.mayBeOverwriteParquetWriteLegacyFormatProp

[ https://issues.apache.org/jira/browse/HUDI-3164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3164: -- Fix Version/s: 0.11.0 0.10.1 > CTAS fails w/ UnsupportedOperationExce

[jira] [Commented] (HUDI-3164) CTAS fails w/ UnsupportedOperationException when trying to modify immutable map in DataSourceUtils.mayBeOverwriteParquetWriteLegacyFormatProp

[ https://issues.apache.org/jira/browse/HUDI-3164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17468638#comment-17468638 ] sivabalan narayanan commented on HUDI-3164: --- CC [~xiaotaotao]  > CTAS fails w/

[jira] [Commented] (HUDI-3164) CTAS fails w/ UnsupportedOperationException when trying to modify immutable map in DataSourceUtils.mayBeOverwriteParquetWriteLegacyFormatProp

[ https://issues.apache.org/jira/browse/HUDI-3164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17468646#comment-17468646 ] tao meng commented on HUDI-3164: [~shivnarayan]   leesf has already solved this problem se

[GitHub] [hudi] hudi-bot commented on pull request #4486: [HUDI-3132] Minor fixes for HoodieCatalog

hudi-bot commented on pull request #4486: URL: https://github.com/apache/hudi/pull/4486#issuecomment-1004870801 ## CI report: * 6d3607ea94783da1e4d0725b3702387dbe685c62 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4486: [HUDI-3132] Minor fixes for HoodieCatalog

hudi-bot removed a comment on pull request #4486: URL: https://github.com/apache/hudi/pull/4486#issuecomment-1003413179 ## CI report: * 6d3607ea94783da1e4d0725b3702387dbe685c62 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4118: [HUDI-2774] Handle duplicate instants while fetching pending clustering plans

hudi-bot commented on pull request #4118: URL: https://github.com/apache/hudi/pull/4118#issuecomment-1004873002 ## CI report: * 3348904bd9d9a0c4469d3bf3c2e2cc47e995ea06 UNKNOWN * 971e52c6cc761bb9b68da6cfa3ba0935db195266 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4118: [HUDI-2774] Handle duplicate instants while fetching pending clustering plans

hudi-bot removed a comment on pull request #4118: URL: https://github.com/apache/hudi/pull/4118#issuecomment-1004825337 ## CI report: * 3348904bd9d9a0c4469d3bf3c2e2cc47e995ea06 UNKNOWN * 971e52c6cc761bb9b68da6cfa3ba0935db195266 Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot commented on pull request #4486: [HUDI-3132] Minor fixes for HoodieCatalog

hudi-bot commented on pull request #4486: URL: https://github.com/apache/hudi/pull/4486#issuecomment-1004873498 ## CI report: * 6d3607ea94783da1e4d0725b3702387dbe685c62 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4486: [HUDI-3132] Minor fixes for HoodieCatalog

hudi-bot removed a comment on pull request #4486: URL: https://github.com/apache/hudi/pull/4486#issuecomment-1004870801 ## CI report: * 6d3607ea94783da1e4d0725b3702387dbe685c62 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] jasondavindev closed issue #4501: [SUPPORT] Duplicate records for same recordKey and differents partitionpath

jasondavindev closed issue #4501: URL: https://github.com/apache/hudi/issues/4501 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr

[GitHub] [hudi] hudi-bot commented on pull request #4486: [HUDI-3132] Minor fixes for HoodieCatalog

hudi-bot commented on pull request #4486: URL: https://github.com/apache/hudi/pull/4486#issuecomment-1004919082 ## CI report: * d96f3e5662350471fd8ff14c47f3daf12e5f151f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4486: [HUDI-3132] Minor fixes for HoodieCatalog

hudi-bot removed a comment on pull request #4486: URL: https://github.com/apache/hudi/pull/4486#issuecomment-1004873498 ## CI report: * 6d3607ea94783da1e4d0725b3702387dbe685c62 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4118: [HUDI-2774] Handle duplicate instants while fetching pending clustering plans

hudi-bot commented on pull request #4118: URL: https://github.com/apache/hudi/pull/4118#issuecomment-1004925986 ## CI report: * 3348904bd9d9a0c4469d3bf3c2e2cc47e995ea06 UNKNOWN * 9e8fcab139fb1fdd9dd2d109b0e73d3e004360c2 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4118: [HUDI-2774] Handle duplicate instants while fetching pending clustering plans

hudi-bot removed a comment on pull request #4118: URL: https://github.com/apache/hudi/pull/4118#issuecomment-1004873002 ## CI report: * 3348904bd9d9a0c4469d3bf3c2e2cc47e995ea06 UNKNOWN * 971e52c6cc761bb9b68da6cfa3ba0935db195266 Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] maddy2u commented on issue #3975: [SUPPORT] Question on hudi's delete statment taking too long

maddy2u commented on issue #3975: URL: https://github.com/apache/hudi/issues/3975#issuecomment-1004932414 > Also, as mentioned in the AWS Blog post, BLOOM Indexes don't scale for our usecase. This is why we moved to SIMPLE Indexes to make it manageable. - https://aws.amazon.com/blo

[jira] [Updated] (HUDI-2774) Async Clustering via deltstreamer fails with IllegalStateException: Duplicate key [==>20211116123724586__replacecommit__INFLIGHT]

[ https://issues.apache.org/jira/browse/HUDI-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-2774: -- Reviewers: sivabalan narayanan > Async Clustering via deltstreamer fails with IllegalStateException: Dup

[jira] [Updated] (HUDI-2774) Async Clustering via deltstreamer fails with IllegalStateException: Duplicate key [==>20211116123724586__replacecommit__INFLIGHT]

[ https://issues.apache.org/jira/browse/HUDI-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-2774: -- Status: In Progress (was: Open) > Async Clustering via deltstreamer fails with IllegalStateException: D

[jira] [Updated] (HUDI-2774) Async Clustering via deltstreamer fails with IllegalStateException: Duplicate key [==>20211116123724586__replacecommit__INFLIGHT]

[ https://issues.apache.org/jira/browse/HUDI-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-2774: -- Reviewers: sivabalan narayanan (was: sivabalan narayanan) > Async Clustering via deltstreamer fails wit

[jira] [Updated] (HUDI-2774) Async Clustering via deltstreamer fails with IllegalStateException: Duplicate key [==>20211116123724586__replacecommit__INFLIGHT]

[ https://issues.apache.org/jira/browse/HUDI-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-2774: -- Status: Patch Available (was: In Progress) > Async Clustering via deltstreamer fails with IllegalStateE

[jira] [Updated] (HUDI-2774) Async Clustering via deltstreamer fails with IllegalStateException: Duplicate key [==>20211116123724586__replacecommit__INFLIGHT]

[ https://issues.apache.org/jira/browse/HUDI-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-2774: -- Reviewers: sivabalan narayanan (was: sivabalan narayanan) > Async Clustering via deltstreamer fails wit

[jira] [Updated] (HUDI-3007) Address minor feedbacks on the repair utility

[ https://issues.apache.org/jira/browse/HUDI-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3007: Fix Version/s: (was: 0.10.1) > Address minor feedbacks on the repair utility > -

[jira] [Assigned] (HUDI-3162) Shade AWS dependencies for bundled packages

[ https://issues.apache.org/jira/browse/HUDI-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-3162: Assignee: Hui An > Shade AWS dependencies for bundled packages > --

[GitHub] [hudi] hudi-bot commented on pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys

hudi-bot commented on pull request #4505: URL: https://github.com/apache/hudi/pull/4505#issuecomment-1005051638 ## CI report: * 15c4b754f2f0a5d085e3b372abbfbba4fa24c69d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys

hudi-bot removed a comment on pull request #4505: URL: https://github.com/apache/hudi/pull/4505#issuecomment-1004684486 ## CI report: * 15c4b754f2f0a5d085e3b372abbfbba4fa24c69d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys

hudi-bot commented on pull request #4505: URL: https://github.com/apache/hudi/pull/4505#issuecomment-1005053857 ## CI report: * 15c4b754f2f0a5d085e3b372abbfbba4fa24c69d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys

hudi-bot removed a comment on pull request #4505: URL: https://github.com/apache/hudi/pull/4505#issuecomment-1005051638 ## CI report: * 15c4b754f2f0a5d085e3b372abbfbba4fa24c69d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[jira] [Commented] (HUDI-3164) CTAS fails w/ UnsupportedOperationException when trying to modify immutable map in DataSourceUtils.mayBeOverwriteParquetWriteLegacyFormatProp

[ https://issues.apache.org/jira/browse/HUDI-3164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17468765#comment-17468765 ] sivabalan narayanan commented on HUDI-3164: --- thanks. > CTAS fails w/ Unsupporte

[jira] [Closed] (HUDI-3164) CTAS fails w/ UnsupportedOperationException when trying to modify immutable map in DataSourceUtils.mayBeOverwriteParquetWriteLegacyFormatProp

[ https://issues.apache.org/jira/browse/HUDI-3164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-3164. - Resolution: Duplicate > CTAS fails w/ UnsupportedOperationException when trying to modify

[GitHub] [hudi] hudi-bot commented on pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

hudi-bot commented on pull request #4352: URL: https://github.com/apache/hudi/pull/4352#issuecomment-1005089234 ## CI report: * 235981abd20a498a3e29e98ce0eda9de35018f99 UNKNOWN * 9ee3e622d98bc3e41ae5ca95d6f5d23b94e45720 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

hudi-bot removed a comment on pull request #4352: URL: https://github.com/apache/hudi/pull/4352#issuecomment-1000177973 ## CI report: * 235981abd20a498a3e29e98ce0eda9de35018f99 UNKNOWN * 9ee3e622d98bc3e41ae5ca95d6f5d23b94e45720 Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot commented on pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys

hudi-bot commented on pull request #4505: URL: https://github.com/apache/hudi/pull/4505#issuecomment-1005100016 ## CI report: * 0979c3974942226999ab7d6f5f92c07724fa1130 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys

hudi-bot removed a comment on pull request #4505: URL: https://github.com/apache/hudi/pull/4505#issuecomment-1005053857 ## CI report: * 15c4b754f2f0a5d085e3b372abbfbba4fa24c69d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] parisni opened a new issue #4506: [SUPPORT] Hive Sync fails silently with embedded derby hive metastore

parisni opened a new issue #4506: URL: https://github.com/apache/hudi/issues/4506 When using a embebded derby database (the spark default), the hive sync does not work silently. The bellow code should create a hive table (hoodie.datasource.hive_sync.enable=true). However, this does

[jira] [Updated] (HUDI-3160) Column Stats index should use the same column for the index key in the write and read code path

[ https://issues.apache.org/jira/browse/HUDI-3160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HUDI-3160: - Status: In Progress (was: Open) > Column Stats index should use the same column for the i

[jira] [Updated] (HUDI-3141) Metadata table getAllFilesInPartition() crashes with NullPointerException

[ https://issues.apache.org/jira/browse/HUDI-3141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HUDI-3141: - Status: Patch Available (was: In Progress) > Metadata table getAllFilesInPartition() cras

[GitHub] [hudi] hudi-bot commented on pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

hudi-bot commented on pull request #4352: URL: https://github.com/apache/hudi/pull/4352#issuecomment-1005121813 ## CI report: * 235981abd20a498a3e29e98ce0eda9de35018f99 UNKNOWN * 9ee3e622d98bc3e41ae5ca95d6f5d23b94e45720 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

hudi-bot removed a comment on pull request #4352: URL: https://github.com/apache/hudi/pull/4352#issuecomment-1005089234 ## CI report: * 235981abd20a498a3e29e98ce0eda9de35018f99 UNKNOWN * 9ee3e622d98bc3e41ae5ca95d6f5d23b94e45720 Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot commented on pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

hudi-bot commented on pull request #4352: URL: https://github.com/apache/hudi/pull/4352#issuecomment-1005162521 ## CI report: * 235981abd20a498a3e29e98ce0eda9de35018f99 UNKNOWN * 83f8b8c57710006349ade4ba712960a2a5d10fca Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

hudi-bot removed a comment on pull request #4352: URL: https://github.com/apache/hudi/pull/4352#issuecomment-1005121813 ## CI report: * 235981abd20a498a3e29e98ce0eda9de35018f99 UNKNOWN * 9ee3e622d98bc3e41ae5ca95d6f5d23b94e45720 Azure: [FAILURE](https://dev.azure.com/apache-hud

  1   2   3   >