chl-wxp opened a new pull request, #10657: URL: https://github.com/apache/seatunnel/pull/10657
## Related Issue Closes: https://github.com/apache/seatunnel/issues/10640 ## What is changed This PR introduces a **Metadata SPI** to provide a unified way to access metadata. Main changes: - Introduce `MetaDataProvider` interface - Support loading providers via Java SPI (`ServiceLoader`) - Add metadata provider configuration in `seatunnel.yaml` This allows different systems to implement metadata access in a consistent and extensible way. ## Backward Compatibility **No backward compatibility is required.** Reason: The Metadata SPI has not been released in any official version, so there are no existing users depending on this API. ## seatunnel.yaml Configuration ```yaml seatunnel: metadata: enabled: true kind: gravitino gravitino: uri: http://127.0.0.1:8090 metalake: test_metalake ``` ## MetaDataProvider Interface ```java public interface MetaDataProvider extends AutoCloseable { /** * Returns a unique identifier for this data source provider. * * <p>The identifier should match the kind specified in the configuration file (e.g., * "gravitino", "datahub", "atlas"). Use lower case for consistency. * * @return unique provider identifier */ String kind(); /** * Initializes the provider with the given configuration. * * @param config the configuration for this provider */ void init(Config config); /** * Maps the given data source ID to connector configuration. * * <p>This method retrieves metadata from the external system for the specified data source and * converts it into a configuration map compatible with the target connector. * * @param connectorIdentifier the connector identifier (e.g., "Jdbc", "MySQL-CDC", "Kafka") * @param metaDataDatasourceId the data source ID in the external metadata system * @return configuration map for the connector, or null if mapping fails */ Map<String, Object> datasourceMap(String connectorIdentifier, String metaDataDatasourceId); /** * Retrieves the table schema for the given metadata table ID. * * <p>This method fetches table metadata from the external metadata system, including column * definitions, data types, and constraints. * * @param metaDataTableId the table ID in the external metadata system * @return table schema if found, empty otherwise */ Optional<TableSchema> tableSchema(String metaDataTableId); /** Closes resources held by this provider. */ @Override void close(); } ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
