Doris-Breakwater commented on issue #66772:
URL: https://github.com/apache/doris/issues/66772#issuecomment-5290714269
Breakwater-GitHub-Analysis-Slot: slot_13a9e214821c
This content is generated by AI for reference only.
Initial assessment: both reported behaviors are supported by the code at
`branch-4.1` commit `0e53b31f586`. Problem 1 is a confirmed Doris
credential-conversion defect and blocks REST credential vending. Problem 2 is a
confirmed Gravitino interoperability defect, although its fix needs to preserve
REST servers that legitimately allow tables directly under the configured
parent namespace. This is not evidence of data corruption or a protocol-version
mismatch.
The issue is currently open with no labels, assignee, milestone, or prior
comments. It is unusually complete: the version, server version/configuration,
REST response, error, reproduction, positive control, and workarounds are
sufficient to start a fix. I did not run the full Doris/Gravitino deployment
locally; the conclusions below come from the exact Doris source, the public
Gravitino v1.3.0 source, and the reporter's controlled reproduction.
Verified facts
1. Vended credentials are lost only on the FE-to-BE conversion path.
- `LanceExternalCatalog.describeTable()` requests credentials for REST
catalogs.
- `loadTableMetadata()` merges `DescribeTableResponse.storage_options`
verbatim into the Java SDK options, explaining why FE metadata reads work.
- The same method calls `LanceStorageOptions.forBackend()`, which
recognizes only `aws_access_key_id`, `aws_secret_access_key`,
`aws_session_token`, `aws_endpoint`, `aws_region`, and
`aws_virtual_hosted_style_request`.
- Gravitino v1.3.0's `LancePropertiesUtils` deliberately strips
`lance.storage.` and therefore returns the configured unprefixed keys such as
`access_key_id`, `secret_access_key`, `endpoint`, and `region`.
- Unmatched vended values are absent from `LanceTableMetadata`;
`LanceScanNode.getLocationProperties()` then sends only that reduced map to the
BE. The BE `lance_reader.cpp` reconstructs Lance options only from normalized
`AWS_*`/`use_path_style` properties, so an empty converted map reaches the
credential provider. This exactly matches the reported FE-success/BE-failure
split and the key-spelling positive control.
2. The unusable root database and failing connectivity check follow directly
from the current namespace mapping.
- `listDatabaseNames()` always publishes `rootDatabase`; that name
decodes to an empty relative namespace and therefore maps to the configured
parent itself.
- `listTableNamesFromRemote()` lists tables against that parent. With
`lance.namespace.parent = lance_catalog`, Doris sends a one-level identifier.
- Gravitino v1.3.0 accepts a one-level identifier for `listNamespaces()`
(catalog -> schemas) but requires exactly two levels for `listTables()`
(catalog + schema), producing the reported error.
- `checkWhenCreating()` calls `listTables(parent)` before
`listNamespaces(parent)`, so `test_connection=true` fails before performing the
valid namespace probe.
- The in-repo REST fixture does not expose this incompatibility: it
permits tables at the root and vends only `aws_*`-prefixed credentials.
What remains a design decision
- The parent namespace is table-listable on the existing fixture and may be
table-listable on other Lance Namespace implementations, so globally removing
the root database would be a compatibility regression. Doris should either
detect parent table-list capability and omit only an unsupported root, or
introduce an explicit mapping policy. Whichever approach is chosen, `SHOW
DATABASES` should not advertise an entry that immediately fails `SHOW TABLES`.
- The connectivity test should first use the operation valid at the
configured parent (`listNamespaces(parent)`). If table-endpoint validation is
retained, it should target a discovered child namespace when one exists rather
than requiring tables to be legal directly under the parent.
Recommended next steps
1. Accept this as a `branch-4.1` Lance REST catalog bug and add the
repository's bug/component labels; the Lance FE code is not present on `master`
at the reported/current revisions.
2. Normalize both accepted S3 key families in one place before building BE
properties. Preserve deterministic precedence when both spellings are returned,
keep vended values ahead of static catalog credentials, and cover the full
temporary-credential set (including the session token) plus
endpoint/region/path-style aliases supported by the pinned Lance object-store
layer.
3. Add focused unit tests for `LanceStorageOptions.forBackend()` covering
unprefixed keys, prefixed keys, conflicts, empty values, static fallback, and
temporary credentials.
4. Extend the REST regression fixture with an unprefixed vending response
and prove a scan succeeds with no static access/secret key. Add a
Gravitino-shaped namespace case where `listNamespaces([catalog])` succeeds,
`listTables([catalog])` is invalid, and `listTables([catalog, schema])`
succeeds; verify both catalog creation with `test_connection=true` and that
every displayed database is listable. Keep a root-table-listable case to
prevent compatibility regressions.
5. Document that `lance.rest.uri` may include the service base path (for
example `/lance`). This is a documentation follow-up, not a blocker for the two
code fixes.
No additional reporter logs or profile are required to begin implementation.
The reporter has offered to submit a PR; maintainers mainly need to choose the
root-database compatibility policy above and review the alias precedence/test
matrix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
