yuw1 opened a new issue, #10445:
URL: https://github.com/apache/gravitino/issues/10445
### What would you like to be improved?
### **Description:**
To leverage the **Time Travel** features of `lance-spark`, we are upgrading
`lance-spark` to the latest version (which depends on `lance-namespace 0.5.2`).
Based on initial integration tests and community discussions, the current
Gravitino Lance REST service (based on version `0.0.20`) has protocol
mismatches with newer clients.
### **Problem Context:**
1. **Missing Table Declaration Endpoint**: The client now requires the
`/declare` endpoint during the table creation flow. According to the Lance
official documentation:
> **"Declares a new Lance table, reserving the table name and location
without creating actual data files."**
> The absence of this route in Gravitino causes a `404 Not Found` error
during table initialization.
2. **Protocol Schema Mismatch**: Newer versions of the protocol have
introduced additional fields (e.g., `context` in `DescribeTableRequest`). The
current server-side deserialization fails with a `400 Bad Request` when
encountering these unrecognized fields:
```
Caused by: java.lang.RuntimeException: Namespace operation failed: Namespace
error: Response error: status=400 Bad Request, content=Unrecognized field
"context" (class com.lancedb.lance.namespace.model.DescribeTableRequest), not
marked as ignorable (2 known properties: "version", "id"])
at [Source:
(org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$UnCloseableInputStream);
line: 1, column: 13] (through reference chain:
com.lancedb.lance.namespace.model.DescribeTableRequest["context"]),
/workspace/rust/lance-namespace-impls/src/rest.rs:577:27
at org.lance.namespace.RestNamespace.describeTableNative(Native Method)
at
org.lance.namespace.RestNamespace.describeTable(RestNamespace.java:180)
at
org.lance.spark.BaseLanceNamespaceSparkCatalog.loadTableInternal(BaseLanceNamespaceSparkCatalog.java:1086)
... 85 more
```
### How should we improve?
1. **Detailed Gap Analysis**: Conduct a thorough code-level audit of the
`lance-namespace` protocol evolution from `0.0.20` to `0.5.x`(and `0.6.x`) to
map out all interface and model changes.
2. **API & Protocol Alignment**:
* Implement missing REST endpoints to support the full table lifecycle
(e.g. `/declare`).
* Update protocol models to align with the parameter changes in existing
APIs.
3. **Backward Compatibility**: Ensure that the upgraded REST service remains
fully compatible with legacy `0.0.20` clients.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
