OIiveirra commented on code in PR #68791:
URL: https://github.com/apache/doris/pull/68791#discussion_r4245202417
##########
fe/fe-connector/fe-connector-fluss/src/main/java/org/apache/doris/connector/fluss/FlussLakeRange.java:
##########
@@ -178,6 +178,12 @@ public Optional<String> getPath() {
return inner.getPath();
}
+ @Override
+ public String getSplitIdentity() {
+ return inner.getSplitIdentity() + "\u0001"
Review Comment:
The lake split identity now delegates to the stable inner file identity, so
changes to the suppressing log tail no longer remap an unchanged lake file.
Added FlussLakeRangeTest coverage. Fixed in 22d6ebafe16.
##########
fe/fe-connector/fe-connector-spi/src/main/java/org/apache/doris/connector/spi/scan/ConnectorScanRange.java:
##########
@@ -46,6 +46,14 @@ default Optional<String> getPath() {
return Optional.empty();
}
+ /**
+ * Returns a stable identity used to assign connector ranges to
external-scan backends. Connectors whose JNI
+ * ranges do not have a path should override this with their serialized
split identity.
+ */
+ default String getSplitIdentity() {
Review Comment:
Bumped the connector plugin API major from 13.0 to 14.0, added
ConnectorScanRange to the frozen surface list and refreshed the baseline. The
pinned API test now verifies 14.0. Fixed in 22d6ebafe16.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/scan/FederationBackendPolicy.java:
##########
@@ -252,7 +269,15 @@ public Multimap<Backend, Split>
computeScanRangeAssignment(List<Split> splits) t
Optional<Backend> chosenNode = candidateNodes.stream()
.min(Comparator.comparingLong(ownerNode ->
assignedWeightPerBackend.get(ownerNode)));
- if (chosenNode.isPresent()) {
+ // A host hint is a local preference for remotely readable
splits. In spread mode, do not
+ // let a repeated hint monopolize one backend while other
eligible backends are idle.
+ boolean localNodeIsWithinGlobalMinimum =
chosenNode.isPresent();
+ if (localNodeIsWithinGlobalMinimum && isSpreadEnabled()) {
+ long minimumAssignedWeight =
Collections.min(assignedWeightPerBackend.values());
Review Comment:
Replaced the per-split Collections.min scan with a maintained TreeMap of
assigned-weight counts, rebuilding it at assignment start and updating it on
every weight change. This keeps the locality check at O(log B) while preserving
the same minimum-weight decision. Fixed in 22d6ebafe16.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
