danny0405 commented on code in PR #13064:
URL: https://github.com/apache/hudi/pull/13064#discussion_r2032210123
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTable.java:
##########
@@ -666,7 +670,7 @@ private void rollbackInflightInstant(HoodieInstant
inflightInstant,
-> entry.getRollbackInstant().getTimestamp())
.orElseGet(HoodieActiveTimeline::createNewInstantTime);
scheduleRollback(context, commitTime, inflightInstant, false,
config.shouldRollbackUsingMarkers(),
- false);
+ false, false);
Review Comment:
> we need to make sure that when clean in scheduled at a given instant time
in data table there isn't a compaction plan on MDT with a greater instant time.
it sounds very restrictive and may break the Flink cleaning workflow, we may
need to skip it for Flink because Flink does not enable MDT in 0.x branch.
> (S1) If Job 2 is ingestion commit and Job 1 is ingestion commit that also
does compaction/log compaction on MDT, then when Job 1 runs before Job 2 and
can create a compaction plan for all instant times (up to ( x ) ) that doesn’t
include instant time (x-1) . Later Job 2 will create instant time (x-1), but
timeline will be in a corrupted state since compaction plan was supposed to
include (x-1)
As for S1, how could the MDT compaction plan being generated when there are
pending instants on DT timeline with smaller timestmap? Should we allow that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
