924060929 commented on code in PR #66711:
URL: https://github.com/apache/doris/pull/66711#discussion_r3775139889
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/FileQueryScanNode.java:
##########
@@ -410,7 +412,7 @@ public void createScanRangeLocations() throws UserException
{
// File splits are generated lazily, and fetched by backends while
scanning.
// Only provide the unique ID of split source to backend.
splitAssignment = new SplitAssignment(backendPolicy, this,
this::splitToScanRange,
- locationProperties, pathPartitionKeys, admissionResult);
+ locationProperties, pathPartitionKeys, admissionResult,
fileAffinitySupported);
Review Comment:
Fixed on the current head by keeping file affinity strictly in the eager
assignment path. The lazy/batch branch now constructs SplitAssignment without
an affinity flag, and SplitAssignment.addToQueue() performs the original
backend assignment without enabling file affinity. Paimon raw-file splits also
defer to the inherited activation flag, so connector-specific ranges cannot
bypass this boundary. Therefore a same-file affinity group can no longer create
the non-owner empty-source condition described here; the existing SplitSource
EOS/timeout protocol is unchanged.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/SplitAssignment.java:
##########
@@ -145,6 +148,7 @@ public void addToQueue(List<Split> splits) throws
UserException {
}
Multimap<Backend, Split> batch = null;
synchronized (assignLock) {
+ enableFileAffinity(splits, fileAffinitySupported);
Review Comment:
Fixed by the same eager-only boundary. Lazy/batch SplitAssignment no longer
enables affinity in addToQueue(), so this PR cannot collapse a lazy file batch
into one backend queue collection and cannot enlarge a fetch response through
the new placement behavior. We intentionally left SplitSource and its
maxBatchSize protocol unchanged; eager assignment does not use that remote
split-source response path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
