the-other-tim-brown commented on code in PR #13347:
URL: https://github.com/apache/hudi/pull/13347#discussion_r2103694645
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/SimpleConcurrentFileWritesConflictResolutionStrategy.java:
##########
@@ -53,24 +56,45 @@ public class
SimpleConcurrentFileWritesConflictResolutionStrategy
@Override
public Stream<HoodieInstant> getCandidateInstants(HoodieTableMetaClient
metaClient, HoodieInstant currentInstant,
Option<HoodieInstant>
lastSuccessfulInstant) {
+ boolean usesCompletionTimeOrdering =
metaClient.getTableConfig().getTableVersion().greaterThanOrEquals(HoodieTableVersion.EIGHT);
HoodieActiveTimeline activeTimeline = metaClient.getActiveTimeline();
// To find which instants are conflicting, we apply the following logic
// 1. Get completed instants timeline only for commits that have happened
since the last successful write.
- // 2. Get any scheduled or completed compaction that have started and/or
finished after the current instant.
+ // 2. For tables with a version lower than 8, get any scheduled or
completed compaction that have started and/or finished after the current
instant.
+ // - Note: this is not required for v8 tables which use completion time
ordering of log files to build file groups.
// 3. Get any completed replace commit that happened since the last
successful write and any pending replace commit.
// We need to check for write conflicts since they may have mutated the
same files that are being newly created by the current write.
+ Predicate<HoodieInstant> compactionInstantFilter = (HoodieInstant instant)
-> {
Review Comment:
I've split this out into two internal methods which seems like the best way
to ensure users automatically pick up these improvements if they upgraded to
table version 8. The diff is also smaller now so it is more clear what is
changing as part of this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
