wangxianghu opened a new pull request, #19081:
URL: https://github.com/apache/hudi/pull/19081
### Describe the issue this Pull Request addresses
Today Hudi ships two partition TTL strategies, both keyed off processing
time:
┌────────────────────────────┬───────────────────────────────────────────────────────────────────────┬──────────────────────────────────────────────────────────────────────────────────┐
│ Strategy │ Reference time
│ Pain point
│
├────────────────────────────┼───────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────┤
│ KeepByCreationTimeStrategy │ Partition creation commit time recorded in
.hoodie_partition_metadata │ Backfilled historical partitions look "newly
created" and never expire │
├────────────────────────────┼───────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────┤
│ KeepByTimeStrategy │ Last commit time touching the partition
│ Any late-arriving write extends the partition's
lifetime, silently disabling TTL │
└────────────────────────────┴───────────────────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────────────────┘
In practice, what most users actually want is event-time semantics: the
date encoded in the partition path (e.g. dt=2026-04-24) is the business event
date, and anything older than N days should be dropped —
regardless of when/how writes land.
This PR introduces a new strategy KeepByEventTimeStrategy that derives the
partition's reference time from the partition path itself, decoupling TTL from
write behavior.
### Summary and Changelog
For users: a new KEEP_BY_EVENT_TIME partition TTL strategy that expires
partitions based on the date encoded in the partition path. TTL is no longer
affected by backfill, late-arriving data, or commit timing.
Supported partition path shapes (v1 scope):
- Day: 2026-04-24 / dt=2026-04-24 / 20260424 / dt=20260424
- Hour: 2026-04-05/06 / dt=2026-04-05/hh=06 / 20260405/06 /
dt=20260405/hh=06
- Prefixed (e.g. eventType=login/20250405/05) via
event.time.partition.start.index
Hive-style is honored from the table's
hoodie.datasource.write.hive_style_partitioning flag (authoritative table-level
setting, not guessed per-segment).
### Impact
- Public API: none broken. New enum value, three new configs (all with
defaults), one new class — strictly additive.
- User-facing: a new opt-in TTL strategy. Existing strategies and configs
are untouched. Users adopt it by setting
hoodie.partition.ttl.management.strategy.type=KEEP_BY_EVENT_TIME.
- Performance: negligible — partition path parsing is in-memory string
ops, runs only inside the existing TTL replace commit, no extra metadata reads.
### Risk Level
low
### Documentation Update
none
### Contributor's checklist
- [ ] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Enough context is provided in the sections above
- [ ] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
