TheR1sing3un commented on code in PR #13347:
URL: https://github.com/apache/hudi/pull/13347#discussion_r2111405579
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/SimpleConcurrentFileWritesConflictResolutionStrategy.java:
##########
@@ -53,12 +55,59 @@ public class
SimpleConcurrentFileWritesConflictResolutionStrategy
@Override
public Stream<HoodieInstant> getCandidateInstants(HoodieTableMetaClient
metaClient, HoodieInstant currentInstant,
Option<HoodieInstant>
lastSuccessfulInstant) {
+ if
(metaClient.getTableConfig().getTableVersion().greaterThanOrEquals(HoodieTableVersion.EIGHT))
{
+ return getCandidateInstantsV8AndAbove(metaClient, currentInstant,
lastSuccessfulInstant);
+ } else {
+ return getCandidateInstantsPreV8(metaClient, currentInstant,
lastSuccessfulInstant);
+ }
+ }
+
+ /**
+ * To find which instants are conflicting for table versions 8 and above, we
apply the following logic:
+ * <ul>
+ * <li>Get completed instants timeline only for commits that have happened
since the last successful write.</li>
+ * <li>Get any completed replace commit that happened since the last
successful write and any pending replace commit.</li>
+ * </ul>
+ * @param metaClient table meta client
+ * @param currentInstant the instant for the write this client is attempting
to commit
+ * @param lastSuccessfulInstant the last successful write before this commit
started
+ * @return a stream of instants that are candidates for conflict resolution
+ */
+ private Stream<HoodieInstant>
getCandidateInstantsV8AndAbove(HoodieTableMetaClient metaClient, HoodieInstant
currentInstant,
+
Option<HoodieInstant> lastSuccessfulInstant) {
+ HoodieActiveTimeline activeTimeline = metaClient.getActiveTimeline();
+ boolean isMoRTable = metaClient.getTableType() ==
HoodieTableType.MERGE_ON_READ;
+ Stream<HoodieInstant> completedCommitsInstantStream = activeTimeline
+ .getCommitsTimeline()
+ .filterCompletedInstants()
+ .filter(instant -> !isMoRTable ||
!instant.getAction().equals(HoodieTimeline.COMMIT_ACTION))
Review Comment:
How about introduce a new method: `HoodieTimeline#getDataCommitsTime`
1. for mor: return [`DELTA_COMMIT`, `REPLACE_COMMIT`]
2. for cow: return [`COMMIT`, `REPLACE_COMMIT`]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
