zhangyue19921010 commented on a change in pull request #4810:
URL: https://github.com/apache/hudi/pull/4810#discussion_r805651637
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java
##########
@@ -492,7 +505,7 @@ protected HoodieBaseFile
addBootstrapBaseFileIfPresent(HoodieFileGroupId fileGro
.map(fileGroup -> Option.fromJavaOptional(fileGroup.getAllBaseFiles()
.filter(baseFile ->
HoodieTimeline.compareTimestamps(baseFile.getCommitTime(),
HoodieTimeline.LESSER_THAN_OR_EQUALS, maxCommitTime
))
- .filter(df ->
!isBaseFileDueToPendingCompaction(df)).findFirst()))
+ .filter(df -> !isBaseFileDueToPendingCompaction(df) &&
!isBaseFileDueToPendingClustering(df)).findFirst()))
Review comment:
When inflight clustering at the earliest instant of the active timeline,
this bug could happen(based on follow code). So that no matter what
maxCommitTime is, we can' t filter it out.
Now we use take containsOrBeforeTimelineStarts as committed, which may
involve unfinished clustering data(this inflight clustering instant is at the
earliest of active timeline.)
https://github.com/apache/hudi/blob/55ecbc662e30068ce0ed49166d254202bd598a8c/hudi-common/src/main/java/org/apache/hudi/common/model/HoodieFileGroup.java#L120
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
