fhan688 commented on PR #19226:
URL: https://github.com/apache/hudi/pull/19226#issuecomment-5018634634
> > > @fhan688 is there any bug we want to fix in this patch?
> >
> >
> > Yes. The bug is that MOR compaction rollback planning can be overly
conservative and leave compaction-produced files behind after the later delta
commits are no longer present.
> > This patch keeps the old base-file-only behavior when later delta
commits still exist, but allows full cleanup of the compaction instant when no
later delta commits remain. The requested-compaction changes are to keep
rollback/restore timeline-only for REQUESTED instants, so they do not generate
file-level rollback work.
>
> didn't get it yet, can you revert the optimization code make make the
smallest change for the fix.
Thanks, understood. I have narrowed this PR down to the compaction rollback
cleanup bug fix only.
The rollback now preserves log files while a later delta commit still
exists, but deletes the full compaction file slice once that later delta
instant has been removed, preventing orphan log files after the compaction base
file is rolled back.
I removed the requested-compaction test because that timeline-only behavior
already exists on the base branch. I also removed the incomplete-instant
metadata handling, partition-pruning, missing-partition tolerance, and
unrelated refactoring changes from this PR.
The focused regression test fails on the PR base and passes with this fix.
The other bug fixes will be submitted separately, and the partition-pruning
optimization will be handled in a follow-up PR with dedicated tests and
supporting evidence.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
