[jira] [Closed] (HUDI-9667) Incorporate completion time into restore workflow

2025-08-04 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9667. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 31dbcfe0896d91d5e753873bb6bbb3f4169fc7

[jira] [Resolved] (HUDI-9667) Incorporate completion time into restore workflow

2025-08-04 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-9667. -- > Incorporate completion time into restore workflow > - > >

[jira] [Updated] (HUDI-9667) Incorporate completion time into restore workflow

2025-08-04 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9667: - Fix Version/s: 1.1.0 > Incorporate completion time into restore workflow > ---

[jira] [Closed] (HUDI-9675) Savepoint safety checks are missing for cleaner policy KEEP_LATEST_FILE_VERSIONS

2025-08-03 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9675. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: aa9935cb998afa9f668f78a221f1c40a8a4368

[jira] [Updated] (HUDI-9675) Savepoint safety checks are missing for cleaner policy KEEP_LATEST_FILE_VERSIONS

2025-08-03 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9675: - Fix Version/s: 1.1.0 > Savepoint safety checks are missing for cleaner policy > KEEP_LATEST_FILE_VERSIONS

[jira] [Closed] (HUDI-9683) Fix spark rename columns for HoodieInternalRowUtils.genUnsafeRowWriter

2025-08-03 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9683. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: f51c36fd64398718c039b26c15767e793b8290

[jira] [Resolved] (HUDI-9683) Fix spark rename columns for HoodieInternalRowUtils.genUnsafeRowWriter

2025-08-03 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-9683. -- > Fix spark rename columns for HoodieInternalRowUtils.genUnsafeRowWriter > -

[jira] [Closed] (HUDI-9676) Avro HoodieAvroUtils.rewriteRecordWithNewSchema logic with renamed columns is wrong

2025-08-02 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9676. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: dc046c4c1b54d9f65a5bc939f027b9509dfad5

[jira] [Closed] (HUDI-9672) Disable skipping clustering for spark incremental query to solve data duplication

2025-08-01 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9672. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 17f3720a48c0b0cecceab2d51fbfa11fcf3168

[jira] [Updated] (HUDI-9671) Ignore -U msg for snapshot queries

2025-08-01 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9671: - Summary: Ignore -U msg for snapshot queries (was: Spark reads the changelog data whose _hoodie_operation

[jira] [Updated] (HUDI-9671) Ignore -U msg for snapshot queries

2025-08-01 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9671: - Fix Version/s: 1.1.0 > Ignore -U msg for snapshot queries > -- > >

[jira] [Closed] (HUDI-9671) Ignore -U msg for snapshot queries

2025-08-01 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9671. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 23f12e1d1a479ceb34cba2b15d230444bda5a1

[jira] [Closed] (HUDI-9673) Remove the eventime metadata function from DefaultHoodieRecordPayload

2025-07-31 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9673. Resolution: Fixed Fixed via master branch: 2869cb70dfb8beadef77ad923407bca56d3b8728 > Remove the eventime m

[jira] [Closed] (HUDI-9630) Support non-globally unique record keys in the record index

2025-07-31 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9630. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: fde04da8904452c27c0aa02a4a09f266eb42bf

[jira] [Assigned] (HUDI-9673) Remove the eventime metadata function from DefaultHoodieRecordPayload

2025-07-31 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen reassigned HUDI-9673: Assignee: Lin Liu > Remove the eventime metadata function from DefaultHoodieRecordPayload > ---

[jira] [Created] (HUDI-9673) Remove the eventime metadata function from DefaultHoodieRecordPayload

2025-07-31 Thread Danny Chen (Jira)
Danny Chen created HUDI-9673: Summary: Remove the eventime metadata function from DefaultHoodieRecordPayload Key: HUDI-9673 URL: https://issues.apache.org/jira/browse/HUDI-9673 Project: Apache Hudi

[jira] [Closed] (HUDI-9661) Fix `Hoodie#getOrderingValue()` to avoid extracting order fields at record level

2025-07-31 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9661. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: e0fa459f1516faf86ff8f718bd347561a8c4bc

[jira] [Updated] (HUDI-9656) revise interface of key encode

2025-07-29 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9656: - Fix Version/s: 1.1.0 > revise interface of key encode > -- > >

[jira] [Commented] (HUDI-9656) revise interface of key encode

2025-07-29 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18010786#comment-18010786 ] Danny Chen commented on HUDI-9656: -- component rename fixed via master branch: 3847b44b0d

[jira] [Updated] (HUDI-9246) Cannot read proto encoded kafka topic from utilities bundle

2025-07-29 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9246: - Fix Version/s: 1.1.0 > Cannot read proto encoded kafka topic from utilities bundle > -

[jira] [Closed] (HUDI-9246) Cannot read proto encoded kafka topic from utilities bundle

2025-07-29 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9246. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 09ecb0c18c897fced60f1383cdf6316bae0c99

[jira] [Closed] (HUDI-9626) Implement pre caching of metadata Hfiles below a certain size threshold

2025-07-29 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9626. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 81ba19813df3ccea1e8b44f90dc1d4b31c0d7d

[jira] [Created] (HUDI-9659) Revisit the design of HoodieRecord metadata

2025-07-28 Thread Danny Chen (Jira)
Danny Chen created HUDI-9659: Summary: Revisit the design of HoodieRecord metadata Key: HUDI-9659 URL: https://issues.apache.org/jira/browse/HUDI-9659 Project: Apache Hudi Issue Type: Improvement

[jira] [Closed] (HUDI-9590) OptimizedLogBlockScan support w/ FG reader

2025-07-28 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9590. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 7e8c7e0016464b1b404ca402633cfa2f6d737c

[jira] [Updated] (HUDI-9590) OptimizedLogBlockScan support w/ FG reader

2025-07-28 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9590: - Fix Version/s: 1.1.0 > OptimizedLogBlockScan support w/ FG reader > --

[jira] [Closed] (HUDI-9650) Add test for buffered record merger

2025-07-27 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9650. Reviewers: Danny Chen, sivabalan narayanan (was: sivabalan narayanan) Resolution: Fixed Fixed via mast

[jira] [Commented] (HUDI-9637) Address comments to an existing PR

2025-07-27 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18010178#comment-18010178 ] Danny Chen commented on HUDI-9637: -- Fixed via master branch: 3f37e656e8115e8821fdbc61d344

[jira] [Closed] (HUDI-9569) Add support for multiple ordering fields

2025-07-26 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9569. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 674acc5fc89b901cb9a60beb12d6ffd8619e54

[jira] [Updated] (HUDI-9569) Add support for multiple ordering fields

2025-07-26 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9569: - Fix Version/s: 1.1.0 > Add support for multiple ordering fields >

[jira] [Closed] (HUDI-9642) Local maven build fails July 24, 2025

2025-07-25 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9642. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 44a15184205014798462b9381345d68de8cbd3

[jira] [Created] (HUDI-9643) Fix `HoodieRecord.getOrderingValue` to be engine agnostic

2025-07-24 Thread Danny Chen (Jira)
Danny Chen created HUDI-9643: Summary: Fix `HoodieRecord.getOrderingValue` to be engine agnostic Key: HUDI-9643 URL: https://issues.apache.org/jira/browse/HUDI-9643 Project: Apache Hudi Issue Typ

[jira] [Updated] (HUDI-9617) Clean redundant adapters for supporting multiple Flink versions

2025-07-24 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9617: - Fix Version/s: 1.1.0 > Clean redundant adapters for supporting multiple Flink versions > -

[jira] [Closed] (HUDI-9617) Clean redundant adapters for supporting multiple Flink versions

2025-07-24 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9617. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: af659920c25734cb11413a7e0dd693c4bebe2f

[jira] [Closed] (HUDI-9226) Add support for Flink 2.0

2025-07-24 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9226. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 14c39f068e3049e62e0ef9ac9f39c5ae4d8dfb

[jira] [Updated] (HUDI-8890) Incremental read with missing/invalid end time will be failed

2025-07-24 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8890: - Fix Version/s: 1.1.0 > Incremental read with missing/invalid end time will be failed > ---

[jira] [Closed] (HUDI-8890) Incremental read with missing/invalid end time will be failed

2025-07-24 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8890. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 99deee6e6dc5c940e143deed2d78f7836e8df4

[jira] [Closed] (HUDI-9612) Improve schema resolver performance by skipping delete-only files

2025-07-23 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9612. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: eb8e829b876d1e670d5651bd6f258ddced6b6d

[jira] [Updated] (HUDI-9612) Improve schema resolver performance by skipping delete-only files

2025-07-23 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9612: - Fix Version/s: 1.1.0 > Improve schema resolver performance by skipping delete-only files > ---

[jira] [Updated] (HUDI-9621) Fix divergence between file index and incremental file index

2025-07-23 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9621: - Fix Version/s: 1.1.0 > Fix divergence between file index and incremental file index >

[jira] [Closed] (HUDI-9621) Fix divergence between file index and incremental file index

2025-07-23 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9621. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 0a24c3c0f6b7437772df3ce49338960083a910

[jira] [Resolved] (HUDI-9621) Fix divergence between file index and incremental file index

2025-07-23 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-9621. -- > Fix divergence between file index and incremental file index > ---

[jira] [Closed] (HUDI-8635) Revisit stats generated in HoodieSparkFileGroupReaderBasedMergeHandle

2025-07-21 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8635. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 6ea3e7dff95dd31a8523ba8bb4e21fb286b76f

[jira] [Closed] (HUDI-9558) Make merge handles configurable

2025-07-21 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9558. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: f93192d446acebb24b7b9379ded3fadc3f3d19

[jira] [Resolved] (HUDI-9592) Use FileGroupReader for metadata table reads in spark datasource

2025-07-21 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-9592. -- > Use FileGroupReader for metadata table reads in spark datasource > ---

[jira] [Closed] (HUDI-9592) Use FileGroupReader for metadata table reads in spark datasource

2025-07-21 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9592. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 21e9a0edc2f3db7f4b351f8fce045ce717e125

[jira] [Updated] (HUDI-9592) Use FileGroupReader for metadata table reads in spark datasource

2025-07-21 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9592: - Fix Version/s: 1.1.0 > Use FileGroupReader for metadata table reads in spark datasource >

[jira] [Updated] (HUDI-9570) Flink AsyncInstant may lost data when trigger recommit.

2025-07-16 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9570: - Fix Version/s: 1.1.0 > Flink AsyncInstant may lost data when trigger recommit. > -

[jira] [Closed] (HUDI-9570) Flink AsyncInstant may lost data when trigger recommit.

2025-07-16 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9570. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 0d0c84705ca31aa8f11d9cce97b83898e4ff23

[jira] [Closed] (HUDI-9512) Empty metadata and timeline files created

2025-07-16 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9512. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: be91a0c11875d83cab041d79878b8311e02b25

[jira] [Updated] (HUDI-9512) Empty metadata and timeline files created

2025-07-16 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9512: - Fix Version/s: 1.1.0 > Empty metadata and timeline files created > --

[jira] [Closed] (HUDI-9575) Skip compaction for file slice with no log files

2025-07-15 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9575. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 6381aacb2fd3225693584ebb4804f43fecf1ac

[jira] [Closed] (HUDI-9451) Avoid broadcasting unnecessary `FileSlice` when reading

2025-07-15 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9451. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: a97be7c74821c3e99f94a77cdc3f7822024bc3

[jira] [Updated] (HUDI-9451) Avoid broadcasting unnecessary `FileSlice` when reading

2025-07-15 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9451: - Fix Version/s: 1.1.0 > Avoid broadcasting unnecessary `FileSlice` when reading > -

[jira] [Closed] (HUDI-9574) Avoid creating InternalSchemaManager for each file in FileGroup reader based Flink Compaction

2025-07-10 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9574. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 60576bcccf08d07abb9d1ee41cf52656fc491b

[jira] [Closed] (HUDI-9571) Avoid scanning timeline to fetch InternalSchema for schema evolution

2025-07-09 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9571. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: aa3dcd83fe660daff4d061be7459ddf02d0386

[jira] [Assigned] (HUDI-9575) Skip compaction for file slice with no log files

2025-07-09 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen reassigned HUDI-9575: Assignee: Shuo Cheng > Skip compaction for file slice with no log files > -

[jira] [Updated] (HUDI-9575) Skip compaction for file slice with no log files

2025-07-09 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9575: - Fix Version/s: 1.1.0 > Skip compaction for file slice with no log files >

[jira] [Created] (HUDI-9575) Skip compaction for file slice with no log files

2025-07-09 Thread Danny Chen (Jira)
Danny Chen created HUDI-9575: Summary: Skip compaction for file slice with no log files Key: HUDI-9575 URL: https://issues.apache.org/jira/browse/HUDI-9575 Project: Apache Hudi Issue Type: Improv

[jira] [Updated] (HUDI-9573) Inspect all the forced class casting in the repo

2025-07-08 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9573: - Description: Currently we see a few type casting errors in a few places: {code:java} org.apache.hudi.commo

[jira] [Created] (HUDI-9573) Inspect all the forced class casting in the repo

2025-07-08 Thread Danny Chen (Jira)
Danny Chen created HUDI-9573: Summary: Inspect all the forced class casting in the repo Key: HUDI-9573 URL: https://issues.apache.org/jira/browse/HUDI-9573 Project: Apache Hudi Issue Type: Improv

[jira] [Closed] (HUDI-9567) BaseHoodieLogRecordReader should skip checking inflight instant for table version EIGHT

2025-07-03 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9567. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 3369b09fd8fca1db1f8e655f79ecb7a97c5736

[jira] [Closed] (HUDI-9564) Inroduce BufferRecordMerger API in FileGroupRecordBuffer

2025-07-03 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9564. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 8aa815ffd28e9dc7758fab3bde040ab1e2fcb3

[jira] [Updated] (HUDI-9332) Pluggable Table Format Support - Interface Integration

2025-07-02 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9332: - Fix Version/s: 1.1.0 > Pluggable Table Format Support - Interface Integration > --

[jira] [Closed] (HUDI-9332) Pluggable Table Format Support - Interface Integration

2025-07-02 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9332. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 8a5163b3aa777cf1b49dab76b3e503a798e80a

[jira] [Updated] (HUDI-9564) Inroduce BufferRecordMerger API in FileGroupRecordBuffer

2025-07-01 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9564: - Description: Currently, we have 5 ramifications for record merging: 1. partial merging; 2. COMMIT_TIME_OR

[jira] [Created] (HUDI-9564) Inroduce BufferRecordMerger API in FileGroupRecordBuffer

2025-07-01 Thread Danny Chen (Jira)
Danny Chen created HUDI-9564: Summary: Inroduce BufferRecordMerger API in FileGroupRecordBuffer Key: HUDI-9564 URL: https://issues.apache.org/jira/browse/HUDI-9564 Project: Apache Hudi Issue Type

[jira] [Updated] (HUDI-9564) Inroduce BufferRecordMerger API in FileGroupRecordBuffer

2025-07-01 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9564: - Fix Version/s: 1.1.0 > Inroduce BufferRecordMerger API in FileGroupRecordBuffer >

[jira] [Updated] (HUDI-9439) Add helpers for reading select fields from schema in FileGroupReader

2025-07-01 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9439: - Fix Version/s: 1.1.0 > Add helpers for reading select fields from schema in FileGroupReader >

[jira] [Closed] (HUDI-9439) Add helpers for reading select fields from schema in FileGroupReader

2025-07-01 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9439. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: e3f93425cd3b5d5fc07d1b39f6396ab403f9b7

[jira] [Closed] (HUDI-9562) Support to use spark.task.cpus to calculate the merge/compaction max memory

2025-06-30 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9562. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 4e9b9287141b067d7d992a1dec0fe8be343550

[jira] [Updated] (HUDI-9562) Support to use spark.task.cpus to calculate the merge/compaction max memory

2025-06-30 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9562: - Fix Version/s: 1.1.0 > Support to use spark.task.cpus to calculate the merge/compaction max memory >

[jira] [Closed] (HUDI-9555) Fix secondary write to metadata table when streaming writes to metadata table is enabled

2025-06-28 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9555. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 33a7b845bcd30f823d026b9f19e20554fe3cf1

[jira] [Commented] (HUDI-8449) Fix deletion of record from FILES partition on empty files list

2025-06-26 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17986470#comment-17986470 ] Danny Chen commented on HUDI-8449: -- final fix via master branch: 205fabba9b6df5227c6a4b49

[jira] [Commented] (HUDI-9340) Add SI support w/ new dag

2025-06-24 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17985995#comment-17985995 ] Danny Chen commented on HUDI-9340: -- Basic pr landed via master: 45312d437a51ccd1d8c75ba0b

[jira] [Updated] (HUDI-9552) Failed to upsert MOR table after setting hoodie.write.set.null.for.missing.columns = true

2025-06-24 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9552: - Description: https://github.com/apache/hudi/issues/13477 > Failed to upsert MOR table after setting > hoo

[jira] [Created] (HUDI-9552) Failed to upsert MOR table after setting hoodie.write.set.null.for.missing.columns = true

2025-06-24 Thread Danny Chen (Jira)
Danny Chen created HUDI-9552: Summary: Failed to upsert MOR table after setting hoodie.write.set.null.for.missing.columns = true Key: HUDI-9552 URL: https://issues.apache.org/jira/browse/HUDI-9552 Project

[jira] [Closed] (HUDI-9535) Prevent Broadcast of full Partitioner during mapPartitionsAsRDD

2025-06-23 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9535. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 1f561d08d7c59d58781ccc2be100eec5b7c145

[jira] [Closed] (HUDI-9527) Use FileGroupReader for HoodieTableMetadataUtil

2025-06-20 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9527. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: e017d85d76b5a2332e96ce0b7e4b2a552f98da

[jira] [Closed] (HUDI-9534) Disable async instant generation for COW table with UPSERT operation

2025-06-20 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9534. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: fcc03c35bc5bf71abf066922fe4ce773e3489b

[jira] [Closed] (HUDI-9531) Avoid generating timestamp again in createCompleteFileInMetaPath

2025-06-20 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9531. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 07d176e2d462d8ff83658caba23d15938d0de7

[jira] [Updated] (HUDI-9531) Avoid generating timestamp again in createCompleteFileInMetaPath

2025-06-20 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9531: - Fix Version/s: 1.1.0 > Avoid generating timestamp again in createCompleteFileInMetaPath >

[jira] [Updated] (HUDI-8881) Potential dataloss in Flink hudi sink

2025-06-18 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8881: - Fix Version/s: 1.1.0 > Potential dataloss in Flink hudi sink > - > >

[jira] [Commented] (HUDI-8881) Potential dataloss in Flink hudi sink

2025-06-18 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17982890#comment-17982890 ] Danny Chen commented on HUDI-8881: -- Fixed via master branch: 74e58bb0d7cb6fe46756879fcdc7

[jira] [Closed] (HUDI-9520) Support new hudi sink based on Flink V2 sink

2025-06-17 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9520. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: cb566b9785fc73a3076a5afabe8ebb38669712

[jira] [Updated] (HUDI-9520) Support new hudi sink based on Flink V2 sink

2025-06-17 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9520: - Fix Version/s: 1.1.0 > Support new hudi sink based on Flink V2 sink >

[jira] [Closed] (HUDI-9524) hudi-defaults.conf does not support include=

2025-06-14 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9524. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 58191bdf5ffbf563386b69a1ac159e1436d47e

[jira] [Updated] (HUDI-9524) hudi-defaults.conf does not support include=

2025-06-14 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9524: - Fix Version/s: 1.1.0 > hudi-defaults.conf does not support include= >

[jira] [Closed] (HUDI-9514) Scanner resources not properly closed in HoodieCompact

2025-06-13 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9514. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: b1af4d4c54636117348b6760b27ae3b7133ceb

[jira] [Updated] (HUDI-9514) Scanner resources not properly closed in HoodieCompact

2025-06-13 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9514: - Fix Version/s: 1.1.0 > Scanner resources not properly closed in HoodieCompact > --

[jira] [Commented] (HUDI-8401) Support CDC payload, partial updates, and custom logic through native record merge API implementation

2025-06-12 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17966786#comment-17966786 ] Danny Chen commented on HUDI-8401: -- FirstValueAvroPayload removed via maste: 1876998d22c

[jira] [Updated] (HUDI-9499) Reduce size of Avro IndexedRecord to decrease spilling probability of FIleGroup reader

2025-06-11 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9499: - Fix Version/s: 1.1.0 > Reduce size of Avro IndexedRecord to decrease spilling probability of > FIleGroup

[jira] [Closed] (HUDI-9499) Reduce size of Avro IndexedRecord to decrease spilling probability of FIleGroup reader

2025-06-11 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9499. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 2d99668881f3f83c224fc787367711ad73e34c

[jira] [Updated] (HUDI-9518) Upgrade Flink 1.15/1.16 parquet version from 1.12.2 to 1.12.3

2025-06-10 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9518: - Fix Version/s: 1.1.0 > Upgrade Flink 1.15/1.16 parquet version from 1.12.2 to 1.12.3 > ---

[jira] [Closed] (HUDI-9518) Upgrade Flink 1.15/1.16 parquet version from 1.12.2 to 1.12.3

2025-06-10 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9518. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 5146dc81f493b4daa06ee4beee21a4f045ad57

[jira] [Updated] (HUDI-9511) Improve performance of timeline server marker handler

2025-06-09 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9511: - Fix Version/s: 1.1.0 > Improve performance of timeline server marker handler > ---

[jira] [Closed] (HUDI-9511) Improve performance of timeline server marker handler

2025-06-09 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9511. Reviewers: Danny Chen, Vinoth Chandar Resolution: Fixed Fixed via master branch: bb3f0796515268b82c6649

[jira] [Closed] (HUDI-9481) Remove extra Netty classes from AWS bundle

2025-06-09 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9481. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 7c6e2e2f20fec5c1fa98f61da119491f9f0c87

[jira] [Updated] (HUDI-9491) Add checkpointId into hudi commit metadata as an extra metadata field

2025-06-08 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-9491: - Fix Version/s: 1.1.0 > Add checkpointId into hudi commit metadata as an extra metadata field > ---

[jira] [Closed] (HUDI-9491) Add checkpointId into hudi commit metadata as an extra metadata field

2025-06-08 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-9491. Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: 01dc67a7e83b4c3274fa2da9369609b8a50bcd

[jira] [Commented] (HUDI-9235) Create a POC for HFile based FG reader

2025-06-06 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-9235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17956679#comment-17956679 ] Danny Chen commented on HUDI-9235: -- iterator refactoring via master: 79863dbf5b8d8b641339

  1   2   3   4   5   6   7   8   9   10   >