924060929 opened a new pull request, #66711: URL: https://github.com/apache/doris/pull/66711
[fix](fe) Keep same-file external splits on one backend ### What problem does this PR solve? Issue Number: close #66601 Related PR: #66602 Problem Summary: External Parquet and ORC files can be divided into multiple scan splits. Assigning those splits to different backends makes each backend open the same remote file and independently read file-level metadata, increasing remote I/O. Keep eligible splits from the same physical file on a stable backend through file-key consistent hashing while preserving every split as an independent scheduling unit. Limit affinity to native Parquet/ORC ranges, including legacy native Paimon raw-file ranges, preserve HDFS block locality and virtual/JNI behavior, and keep affinity splits out of load redistribution. ### Release note Improve external Parquet and ORC split placement by keeping ranges of the same file on one backend. ### Check List (For Author) - Test: Not run locally because this branch predates worktree initialization support and the isolated worktree has no `thirdparty/installed` toolchain. - Behavior changed: Yes. Eligible ranges of one remote Parquet/ORC file are assigned to one backend while remaining separate scan ranges. - Does this need documentation: No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
