nsivabalan opened a new pull request, #5350:
URL: https://github.com/apache/hudi/pull/5350
## What is the purpose of the pull request
- So far, archival will stop at the first savepointed commit and will not
proceed further. For users who may not be interested in incremental queries,
but just "as of instant", we can let them proceed further by skipping just the
savepointed commit. This opens up new opportunities for hudi users. For eg, one
can retain commits for years, by adding one savepoint per day for older commits
(say > 30 days old). And they can query hudi using "as of instant" for very
old data. If not, one has to retain every commit and let archival stop at first
commit which may not be a good experience for users.
## Brief change log
- Added a new config named `hoodie.archive.proceed.beyond.savepoints` which
will guard this behavior. Have set the default value to true.
- When this config is enabled, archival will skip savepointed commits and
will proceed further to archive other commits. when disabled archival will stop
at first savepointed commit.
## Verify this pull request
This change added tests and can be verified as follows:
- TestHoodietTimelineArchiver.testSavepointsWithArchival
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under
an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
