jarredhj0214 opened a new issue, #13552:
URL: https://github.com/apache/gravitino/issues/13552
### What would you like to be improved?
Apache Lindorm exposes a MySQL-compatible endpoint and can be accessed with
MySQL Connector/J. However, its SQL dialect, metadata tables, and permission
model are not fully compatible with MySQL.
When a Lindorm endpoint is configured as a `jdbc-mysql` catalog, Gravitino
fails to load some tables while reading index metadata. Operations depending on
`loadTable`, including GRANT, fail as a result.
The failure occurs in `JdbcTableOperations#getIndexes` when it invokes
`DatabaseMetaData#getIndexInfo`. MySQL Connector/J 8.0.33 generates a query
containing:
```sql
SELECT ..., CARDINALITY, ...
FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = ? AND TABLE_NAME = ?
```
Lindorm rejects the unquoted `CARDINALITY` token:
```text
ERROR 1064 (42000): You have an error in your SQL syntax;
Incorrect syntax near the keyword 'CARDINALITY'
```
The following behavior was verified against a Lindorm MySQL endpoint:
- `SELECT VERSION()` reports `8.0.28`.
- `@@version_comment` is `NULL`.
- `INFORMATION_SCHEMA.STATISTICS` returns no rows when queried without
`CARDINALITY`.
- `SHOW INDEX` fails for a read-only account because Lindorm requires
`ADMIN`.
- `INFORMATION_SCHEMA.KEY_COLUMN_USAGE` correctly returns the primary key.
- `DESCRIBE TABLE` works with `READ` permission and returns
`IS_PRIMARY_KEY`, `SORT_ORDER`, `IS_AUTO_INCREMENT`, and `IS_LOCAL_UNIQUE`.
A representative Lindorm table is:
```sql
CREATE TABLE db_name.example_table (
id BIGINT NOT NULL,
value VARCHAR,
PRIMARY KEY (id)
);
```
This is not caused by a user column named `CARDINALITY`. It is caused by
using MySQL metadata SQL against a database that supports the MySQL wire
protocol but has different metadata semantics.
Lindorm documents that its MySQL protocol endpoint is not fully compatible
with MySQL SQL:
https://help.aliyun.com/zh/lindorm/user-guide/mysql-protocol-development-instructions/
Its `DESCRIBE TABLE` command exposes primary-key metadata with `READ`
permission:
https://help.aliyun.com/zh/lindorm/developer-reference/describe-3
`SHOW INDEX` requires both `READ` and `ADMIN`:
https://help.aliyun.com/zh/lindorm/developer-reference/ddl-command-overview
In an existing unified metadata model, Lindorm endpoints that expose MySQL
compatibility are classified as the MySQL data source type. They are therefore
registered in Gravitino with the `jdbc-mysql` provider and MySQL Connector/J.
Introducing a separate `jdbc-lindorm` provider would require adding a new
data source type, changing provider mappings, migrating existing catalog
registrations, and updating downstream integrations.
### How should we improve?
We would like to discuss which integration model is preferred.
#### Option 1: Add a dedicated `jdbc-lindorm` catalog
Add a Lindorm provider under `catalogs-contrib`, similar to the existing
OceanBase and Hologres catalogs.
The provider could:
- Continue using MySQL Connector/J and `jdbc:mysql://` URLs.
- Use Lindorm-specific metadata SQL such as `DESCRIBE TABLE`.
- Model Lindorm's type system and DDL capabilities explicitly.
- Initially support read-only metadata operations required by authorization.
- Avoid MySQL-specific operations such as `SHOW TABLE STATUS` and
`DatabaseMetaData#getIndexInfo`.
This provides a clean dialect boundary, but requires a new data source type
and updates to existing registrations and downstream integrations. Spark
currently handles generic `jdbc-*` providers, while Flink and Trino have
explicit provider registrations.
#### Option 2: Add a Lindorm metadata dialect to `jdbc-mysql`
Keep the MySQL data source classification and introduce an explicit property
such as:
```properties
jdbc-dialect=lindorm
```
The MySQL catalog would select Lindorm-specific metadata operations when
this property is configured.
This preserves the current registration model and has a smaller ecosystem
impact. However, it mixes Lindorm behavior into the MySQL provider and may
become difficult to maintain as more differences are discovered.
#### Possible limited fallback
As a narrow compatibility fix, Gravitino could preserve primary keys
returned by `getPrimaryKeys()` and skip unique-index metadata when
`getIndexInfo()` fails with SQLState `42000`, error code `1064`, and an error
mentioning `CARDINALITY`.
This would unblock `loadTable` and GRANT, but additional unique-index
metadata could be omitted. It also does not address other potential MySQL
dialect differences.
Questions for the community:
1. Should protocol-compatible databases remain under `jdbc-mysql` with an
explicit metadata dialect or capability strategy?
2. Is a dedicated provider appropriate when existing systems classify
Lindorm as the MySQL data source type?
3. Would a read-only metadata implementation be an acceptable initial scope?
4. Should optional index metadata ever prevent a JDBC table from being
loaded?
5. Should Gravitino provide a generic fallback that preserves primary keys
while omitting unavailable unique-index metadata?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
