FANNG1 commented on issue #13065:
URL: https://github.com/apache/gravitino/issues/13065#issuecomment-5657972859
Confirming this against a live setup, and adding a data point that I think
reframes the question a little.
## Confirmed
Gravitino 1.3.0 with the `lance-rest` auxiliary service on `:9101`, backed
by a `lakehouse-generic` catalog on MinIO. The non-distributed `VECTOR_SEARCH`
path goes through `namespace.queryTable(...)`, and the request shows up at the
server as:
```
POST /lance/v1/table/lance_spark_vs$vs$vec/query?delimiter=%24 -> 404
```
So the report is accurate: `LanceTableOperations` implements
describe/create/declare/register/deregister/exists/drop/drop_columns/alter_columns,
and there is no `query` route.
## The part that may change the framing
The distributed execution path in lance-spark **does not call `queryTable`
at all**. The driver plans partitions by opening the dataset directly and each
task scans its own unit, so the namespace only has to answer `describeTable`.
With that path enabled the same `VECTOR_SEARCH` query works end to end against
an unmodified Gravitino Lance REST server.
I put a proxy in front of `:9101` and logged every request across a full
test run — creating tables, writing, building vector indexes, and running
distributed searches. lance-spark touched exactly six endpoints:
```
150 /v1/table/{id}/describe 16 /v1/namespace/{id}/list
4 /v1/table/{id}/exists 2 /v1/table/{id}/declare
1 /v1/namespace/{id}/table/list 1 /v1/namespace/{id}/create
```
Every one of those is already implemented. The only 404 in the whole run was
the single `query` call from the non-distributed path.
Credential vending works as-is too: `describeTable` returns
`storage_options` with `access_key_id` / `secret_access_key` / `endpoint` /
`region` / `allow_http`, and Spark needs no storage configuration of its own. I
verified that by breaking the catalog's `lance.storage.secret_access_key` — the
tasks then fail with `SignatureDoesNotMatch` / 403 from S3 and recover once it
is restored, so the credentials demonstrably come from Gravitino and nowhere
else.
What was verified end to end, with zero changes on the Gravitino side:
| scenario | result |
|---|---|
| non-distributed `VECTOR_SEARCH` | fails on the `query` 404 described in
this issue |
| distributed, no index (per-fragment KNN) | correct top-k |
| distributed, separate executor JVMs (`local-cluster`) | identical results |
| distributed, one IVF_FLAT segment per fragment | one task per segment,
correct top-k |
| mixed: some fragments indexed, some bare | no duplicates, no missing rows |
| `fast_search => true` | unindexed fragments correctly excluded |
| filter / offset / `k` larger than row count | correct |
| executor-side credential refresh on / off | 4 fragments → 4 distinct
credential fetches / driver only |
Also worth recording for #10445: a **lance-namespace 0.11.1 client against
this 0.7.5 server** worked for every operation above, and both sides default to
the `$` delimiter.
## So: is `queryTable` on the server still worth it?
I think it is a real design question rather than a straightforward gap, and
it is worth discussing explicitly:
- Implementing `queryTable` would make the Gravitino Lance REST server a
**data-plane** component — it would have to open the dataset, run the ANN
search and return Arrow IPC. That is a different operational profile from
serving metadata and credentials: it needs object-store bandwidth, memory for
the search, and it becomes a scaling bottleneck for large top-k over large
tables.
- The distributed path keeps Gravitino on the metadata plane and pushes
execution to the engine, which parallelises naturally and reuses the
credentials Gravitino already vends.
- On the other hand, a server-side `queryTable` is the only option for thin
clients that cannot reach the object store themselves, and it is part of the
Lance Namespace spec, so leaving it unimplemented is a visible conformance gap.
The error surface is also poor today: the client reports `Internal error:
Failed to parse error response: status=404 Not Found, body=<html>` because it
tries to parse Jetty's HTML error page, rather than saying the operation is
unsupported.
A cheap intermediate step, regardless of which way the bigger decision goes,
would be returning a structured "operation not supported" error for
unimplemented routes so clients get an actionable message.
## Status
https://github.com/FANNG1/lance-spark/pull/1 stacks the three lance-spark
pieces needed to make this work — the Lance 12 dependency upgrade, distributed
`VECTOR_SEARCH` (#608), and a fix for a filter-semantics bug found while
testing it — and is what the results above were produced with. I will follow up
on getting the upstream PRs merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
