github-actions[bot] commented on code in PR #66227:
URL: https://github.com/apache/doris/pull/66227#discussion_r4134849764
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/paimon/source/PaimonScanNode.java:
##########
@@ -348,6 +385,283 @@ private void setScanLevelPaimonOptions() {
}
}
+ /**
+ * Whether the table location's URI scheme is served by a paimon-rust
FileIO parser whose
+ * property translation the FE/BE bridge implements (the s3.* / fs.oss.*
key families for
+ * s3 / s3a / oss) or that needs no credentials at all (hdfs hadoop conf
and local
+ * filesystem paths). Every other scheme the pinned crate dispatches to
its own parser
+ * (cosn / obs / gs / abfs and friends) must fall back to the JNI reader
because Doris
+ * delivers those credentials only as AWS_* aliases that those parsers do
not read.
+ * A null location cannot be verified (and cannot ship paimon_table
either), so it is
+ * not rust-eligible.
+ *
+ * <p>The scheme must also appear in the exact lowercase form the pinned
crate consumes.
+ * URI schemes are case-insensitive, but the crate lowercases only its
storage
+ * dispatch: its object-store path extraction strips a lowercase {@code
s3://} prefix
+ * from the original string, and the hdfs / file helpers likewise match
only lowercase
+ * prefixes. A {@code S3://} or {@code Hdfs://} warehouse also produces
DataSplit file
+ * paths in that original casing (serialized by the paimon SDK before the
FE sees
+ * them), so the mixed-case shape fails the rust open beyond the
transported location.
+ * Those valid URI variants must therefore route to JNI, whose Java stack
is
+ * case-insensitive everywhere.
+ */
+ @VisibleForTesting
+ static boolean isRustVerifiedLocationScheme(String location) {
+ if (location == null) {
+ return false;
+ }
+ int sep = location.indexOf("://");
+ if (sep <= 0) {
Review Comment:
[P1] Keep authority-less HDFS locations on JNI. A Paimon warehouse at
`hdfs:/warehouse` with `fs.defaultFS=hdfs://nn:8020` is valid for the Java
reader: Hadoop supplies the missing authority. This `://` check treats it as a
local path and selects Rust, whose pinned HDFS parser requires `hdfs://` and
fails the scan. Detect the actual URI scheme and route this shape to JNI unless
it is normalized for the Rust table and split paths; add a routing test for the
single-slash form.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
