[jira] [Created] (HUDI-9634) Archival considers retaining the `the earliest retain instant` in the clean plan

Chaoyang Liu created HUDI-9634: -- Summary: Archival considers retaining the `the earliest retain instant` in the clean plan Key: HUDI-9634 URL: https://issues.apache.org/jira/browse/HUDI-9634 Project: Apa

[jira] [Created] (HUDI-9554) Ensure the order of the partition schema of the query output

Chaoyang Liu created HUDI-9554: -- Summary: Ensure the order of the partition schema of the query output Key: HUDI-9554 URL: https://issues.apache.org/jira/browse/HUDI-9554 Project: Apache Hudi I

[jira] [Created] (HUDI-9547) Make the Hudi-CLI compatible with the new changes in 1.x

Chaoyang Liu created HUDI-9547: -- Summary: Make the Hudi-CLI compatible with the new changes in 1.x Key: HUDI-9547 URL: https://issues.apache.org/jira/browse/HUDI-9547 Project: Apache Hudi Issue

[jira] [Created] (HUDI-9466) RFC for Unified Bucket Index

Chaoyang Liu created HUDI-9466: -- Summary: RFC for Unified Bucket Index Key: HUDI-9466 URL: https://issues.apache.org/jira/browse/HUDI-9466 Project: Apache Hudi Issue Type: New Feature

[jira] [Created] (HUDI-9451) Avoid broadcasting unnecessary `FileSlice` when reading

Chaoyang Liu created HUDI-9451: -- Summary: Avoid broadcasting unnecessary `FileSlice` when reading Key: HUDI-9451 URL: https://issues.apache.org/jira/browse/HUDI-9451 Project: Apache Hudi Issue T

[jira] [Created] (HUDI-9450) Optimize the string-related performance on the hot path

Chaoyang Liu created HUDI-9450: -- Summary: Optimize the string-related performance on the hot path Key: HUDI-9450 URL: https://issues.apache.org/jira/browse/HUDI-9450 Project: Apache Hudi Issue T

[jira] [Created] (HUDI-9436) The file slice is filtered if the first log in slice is still in pending

Chaoyang Liu created HUDI-9436: -- Summary: The file slice is filtered if the first log in slice is still in pending Key: HUDI-9436 URL: https://issues.apache.org/jira/browse/HUDI-9436 Project: Apache Hudi

[jira] [Created] (HUDI-9330) Avoid storing the clean plan in inflight clean instant

Chaoyang Liu created HUDI-9330: -- Summary: Avoid storing the clean plan in inflight clean instant Key: HUDI-9330 URL: https://issues.apache.org/jira/browse/HUDI-9330 Project: Apache Hudi Issue Ty

[jira] [Created] (HUDI-9329) return an illegal partition id after `BucketPartitionUtils::createDataFrame`

Chaoyang Liu created HUDI-9329: -- Summary: return an illegal partition id after `BucketPartitionUtils::createDataFrame` Key: HUDI-9329 URL: https://issues.apache.org/jira/browse/HUDI-9329 Project: Apache

[jira] [Created] (HUDI-9268) Dynamically computes compaction/merge available memory and passes it to `FileGroupReader`

Chaoyang Liu created HUDI-9268: -- Summary: Dynamically computes compaction/merge available memory and passes it to `FileGroupReader` Key: HUDI-9268 URL: https://issues.apache.org/jira/browse/HUDI-9268 Pro

[jira] [Created] (HUDI-9302) enable vectorized reading for file slice without log file

Chaoyang Liu created HUDI-9302: -- Summary: enable vectorized reading for file slice without log file Key: HUDI-9302 URL: https://issues.apache.org/jira/browse/HUDI-9302 Project: Apache Hudi Issue

[jira] [Created] (HUDI-9250) Introduce a representative file containing the estimated total size of file slice

Chaoyang Liu created HUDI-9250: -- Summary: Introduce a representative file containing the estimated total size of file slice Key: HUDI-9250 URL: https://issues.apache.org/jira/browse/HUDI-9250 Project: Ap

[jira] [Created] (HUDI-9248) Unify code paths for all write operations about `bulk_insert`

Chaoyang Liu created HUDI-9248: -- Summary: Unify code paths for all write operations about `bulk_insert` Key: HUDI-9248 URL: https://issues.apache.org/jira/browse/HUDI-9248 Project: Apache Hudi

[jira] [Created] (HUDI-9202) Introduce extensible bucket index in NBCC mode

Chaoyang Liu created HUDI-9202: -- Summary: Introduce extensible bucket index in NBCC mode Key: HUDI-9202 URL: https://issues.apache.org/jira/browse/HUDI-9202 Project: Apache Hudi Issue Type: New

[jira] [Created] (HUDI-9125) Maximum memory for merging do not take effect when compaction using file group reader based mode

Chaoyang Liu created HUDI-9125: -- Summary: Maximum memory for merging do not take effect when compaction using file group reader based mode Key: HUDI-9125 URL: https://issues.apache.org/jira/browse/HUDI-9125

[jira] [Created] (HUDI-9169) Data loss in insert/delete/ insert-again cases

Chaoyang Liu created HUDI-9169: -- Summary: Data loss in insert/delete/ insert-again cases Key: HUDI-9169 URL: https://issues.apache.org/jira/browse/HUDI-9169 Project: Apache Hudi Issue Type: Bug

[jira] [Created] (HUDI-9166) Introduce schema pruning for delete-record

Chaoyang Liu created HUDI-9166: -- Summary: Introduce schema pruning for delete-record Key: HUDI-9166 URL: https://issues.apache.org/jira/browse/HUDI-9166 Project: Apache Hudi Issue Type: Improvem

[jira] [Created] (HUDI-9152) Improve read/write/compaction performance by reducing the overhead of avro-schema comparison

Chaoyang Liu created HUDI-9152: -- Summary: Improve read/write/compaction performance by reducing the overhead of avro-schema comparison Key: HUDI-9152 URL: https://issues.apache.org/jira/browse/HUDI-9152

[jira] [Created] (HUDI-9025) Improve append performance by reducing schema comparisons

Chaoyang Liu created HUDI-9025: -- Summary: Improve append performance by reducing schema comparisons Key: HUDI-9025 URL: https://issues.apache.org/jira/browse/HUDI-9025 Project: Apache Hudi Issue

[jira] [Created] (HUDI-8892) Introduce projection push down for payload mode

Chaoyang Liu created HUDI-8892: -- Summary: Introduce projection push down for payload mode Key: HUDI-8892 URL: https://issues.apache.org/jira/browse/HUDI-8892 Project: Apache Hudi Issue Type: New

[jira] [Created] (HUDI-8890) Incremental read with missing/invalid end time will be failed

Chaoyang Liu created HUDI-8890: -- Summary: Incremental read with missing/invalid end time will be failed Key: HUDI-8890 URL: https://issues.apache.org/jira/browse/HUDI-8890 Project: Apache Hudi

[jira] [Created] (HUDI-8889) Trim unnecessary columns during MoR snapshot read

Chaoyang Liu created HUDI-8889: -- Summary: Trim unnecessary columns during MoR snapshot read Key: HUDI-8889 URL: https://issues.apache.org/jira/browse/HUDI-8889 Project: Apache Hudi Issue Type: I

[jira] [Updated] (HUDI-8866) Unified the file naming rules in NBCC mode

[ https://issues.apache.org/jira/browse/HUDI-8866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyang Liu updated HUDI-8866: --- Summary: Unified the file naming rules in NBCC mode (was: Unified the file naming rules on Spark in N

[jira] [Created] (HUDI-8866) Unified the file naming rules on Spark in NBCC mode

Chaoyang Liu created HUDI-8866: -- Summary: Unified the file naming rules on Spark in NBCC mode Key: HUDI-8866 URL: https://issues.apache.org/jira/browse/HUDI-8866 Project: Apache Hudi Issue Type:

[jira] [Closed] (HUDI-8838) Wrong way to get the latest completed instant

[ https://issues.apache.org/jira/browse/HUDI-8838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyang Liu closed HUDI-8838. -- Resolution: Not A Bug > Wrong way to get the latest completed instant >

[jira] [Created] (HUDI-8838) Wrong way to get the latest completed instant

Chaoyang Liu created HUDI-8838: -- Summary: Wrong way to get the latest completed instant Key: HUDI-8838 URL: https://issues.apache.org/jira/browse/HUDI-8838 Project: Apache Hudi Issue Type: Bug

[jira] [Created] (HUDI-8802) avoid unnecessary conversion between HoodieIndexAvroRecord and HoodieAvroRecord in HoodieFileSliceReader

Chaoyang Liu created HUDI-8802: -- Summary: avoid unnecessary conversion between HoodieIndexAvroRecord and HoodieAvroRecord in HoodieFileSliceReader Key: HUDI-8802 URL: https://issues.apache.org/jira/browse/HUDI-8802

[jira] [Created] (HUDI-8800) Introduce single-spark-job clustering for consistent bucket to improve performance

Chaoyang Liu created HUDI-8800: -- Summary: Introduce single-spark-job clustering for consistent bucket to improve performance Key: HUDI-8800 URL: https://issues.apache.org/jira/browse/HUDI-8800 Project: A

[jira] [Created] (HUDI-8794) The log file is not properly closed when the task is killed

Chaoyang Liu created HUDI-8794: -- Summary: The log file is not properly closed when the task is killed Key: HUDI-8794 URL: https://issues.apache.org/jira/browse/HUDI-8794 Project: Apache Hudi Is

[jira] [Updated] (HUDI-8787) improve compaction performance by reducing unnecessary disk access

[ https://issues.apache.org/jira/browse/HUDI-8787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyang Liu updated HUDI-8787: --- Summary: improve compaction performance by reducing unnecessary disk access (was: improve compaction

[jira] [Created] (HUDI-8787) improve compaction performance by reducing unnecessary memory usage and unnecessary disk access

Chaoyang Liu created HUDI-8787: -- Summary: improve compaction performance by reducing unnecessary memory usage and unnecessary disk access Key: HUDI-8787 URL: https://issues.apache.org/jira/browse/HUDI-8787

[jira] [Created] (HUDI-8781) optimize executor memory usage during executing clustering

Chaoyang Liu created HUDI-8781: -- Summary: optimize executor memory usage during executing clustering Key: HUDI-8781 URL: https://issues.apache.org/jira/browse/HUDI-8781 Project: Apache Hudi Issu

[jira] [Created] (HUDI-8678) improve consistent-bucket resizing performance by reducing unnecessary record collecting

Chaoyang Liu created HUDI-8678: -- Summary: improve consistent-bucket resizing performance by reducing unnecessary record collecting Key: HUDI-8678 URL: https://issues.apache.org/jira/browse/HUDI-8678 Proj

[jira] [Created] (HUDI-8676) improve ValidationUtils performance by lazy appending msg

Chaoyang Liu created HUDI-8676: -- Summary: improve ValidationUtils performance by lazy appending msg Key: HUDI-8676 URL: https://issues.apache.org/jira/browse/HUDI-8676 Project: Apache Hudi Issue

[jira] [Created] (HUDI-8626) Incomplete state of the file of consistent-hash-metadata

Chaoyang Liu created HUDI-8626: -- Summary: Incomplete state of the file of consistent-hash-metadata Key: HUDI-8626 URL: https://issues.apache.org/jira/browse/HUDI-8626 Project: Apache Hudi Issue

[jira] [Created] (HUDI-8622) fix performance regression of tag when written into consistent bucket index table

Chaoyang Liu created HUDI-8622: -- Summary: fix performance regression of tag when written into consistent bucket index table Key: HUDI-8622 URL: https://issues.apache.org/jira/browse/HUDI-8622 Project: Ap

[jira] [Created] (HUDI-8590) Wrong file path for commit-marker-file in Consistent-Bucket-Index layout

Chaoyang Liu created HUDI-8590: -- Summary: Wrong file path for commit-marker-file in Consistent-Bucket-Index layout Key: HUDI-8590 URL: https://issues.apache.org/jira/browse/HUDI-8590 Project: Apache Hudi

[jira] [Created] (HUDI-8565) Avoid RetryHelper throw Exception because of Long Type overflow

Chaoyang Liu created HUDI-8565: -- Summary: Avoid RetryHelper throw Exception because of Long Type overflow Key: HUDI-8565 URL: https://issues.apache.org/jira/browse/HUDI-8565 Project: Apache Hudi

[jira] [Created] (HUDI-8488) Introduce SLF4J metrics reporter

Chaoyang Liu created HUDI-8488: -- Summary: Introduce SLF4J metrics reporter Key: HUDI-8488 URL: https://issues.apache.org/jira/browse/HUDI-8488 Project: Apache Hudi Issue Type: Improvement

[jira] [Created] (HUDI-8482) Introduce partition-level metrics

Chaoyang Liu created HUDI-8482: -- Summary: Introduce partition-level metrics Key: HUDI-8482 URL: https://issues.apache.org/jira/browse/HUDI-8482 Project: Apache Hudi Issue Type: Improvement

[jira] [Created] (HUDI-8393) Supporting a query task can read multiple file-slice to reduce spark task num

Chaoyang Liu created HUDI-8393: -- Summary: Supporting a query task can read multiple file-slice to reduce spark task num Key: HUDI-8393 URL: https://issues.apache.org/jira/browse/HUDI-8393 Project: Apache

[jira] [Created] (HUDI-8382) Improve MOR-Snapshot-Query performance for COW like table

Chaoyang Liu created HUDI-8382: -- Summary: Improve MOR-Snapshot-Query performance for COW like table Key: HUDI-8382 URL: https://issues.apache.org/jira/browse/HUDI-8382 Project: Apache Hudi Issue

[jira] [Created] (HUDI-8238) Missing logs in need when scan logs for mor read

Chaoyang Liu created HUDI-8238: -- Summary: Missing logs in need when scan logs for mor read Key: HUDI-8238 URL: https://issues.apache.org/jira/browse/HUDI-8238 Project: Apache Hudi Issue Type: Bu

[jira] [Created] (HUDI-8215) Support composition compaction strategy

Chaoyang Liu created HUDI-8215: -- Summary: Support composition compaction strategy Key: HUDI-8215 URL: https://issues.apache.org/jira/browse/HUDI-8215 Project: Apache Hudi Issue Type: New Feature

[jira] [Created] (HUDI-8214) Support specify partitions with regex for compaction

Chaoyang Liu created HUDI-8214: -- Summary: Support specify partitions with regex for compaction Key: HUDI-8214 URL: https://issues.apache.org/jira/browse/HUDI-8214 Project: Apache Hudi Issue Type

[jira] [Commented] (HUDI-8114) Introduce in-memory-cache for ExternalSpillableMap to improve performance

[ https://issues.apache.org/jira/browse/HUDI-8114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17875792#comment-17875792 ] Chaoyang Liu commented on HUDI-8114: Hi, I am working on it now, please assign it to m

[jira] [Created] (HUDI-8084) Sort Merge Join Compaction

Chaoyang Liu created HUDI-8084: -- Summary: Sort Merge Join Compaction Key: HUDI-8084 URL: https://issues.apache.org/jira/browse/HUDI-8084 Project: Apache Hudi Issue Type: Improvement Co

[jira] [Updated] (HUDI-8084) Support Sort Merge Join Compaction

[ https://issues.apache.org/jira/browse/HUDI-8084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyang Liu updated HUDI-8084: --- Summary: Support Sort Merge Join Compaction (was: Sort Merge Join Compaction) > Support Sort Merge J