github-actions[bot] commented on code in PR #66710:
URL: https://github.com/apache/doris/pull/66710#discussion_r3781124381
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/split/PluginDrivenSplit.java:
##########
@@ -68,6 +69,16 @@ public Object getInfo() {
return null;
}
+ @Override
+ public Optional<String> getFileAffinityKey() {
+ String fileFormat = connectorScanRange.getFileFormat();
+ boolean supportedFormat = "parquet".equalsIgnoreCase(fileFormat) ||
"orc".equalsIgnoreCase(fileFormat);
+ return isFileAffinitySupported() &&
connectorScanRange.isNativeReadRange() && supportedFormat
+ && getHosts().length == 0 && getFileLength() > getLength()
+ ? connectorScanRange.getPath().map(path ->
LocationPath.of(path).getNormalizedLocation())
Review Comment:
`getFileAffinityKey()` is called during eligibility, consistent-hash
funneling, redistribution fencing, and scan-range materialization, so an
eligible split reparses this same URI four times after the constructor has
already parsed it. One-argument `LocationPath.of` stores the original string
unchanged as its normalized value, and built-in connector providers normalize
before constructing the range. This adds avoidable allocations to the large
sliced scans this PR targets. Please reuse the existing connector path while
preserving its optional semantics.
```suggestion
? connectorScanRange.getPath()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
