FANNG1 opened a new pull request, #10501:
URL: https://github.com/apache/gravitino/pull/10501
### What changes were proposed in this pull request?
This PR refactors the Gravitino Flink connector multi-version layout to
follow a Spark-like structure for Flink `1.18`, `1.19`, and `1.20`.
The main changes are:
- add version-specific catalog entry classes and factory classes in `v1.18`,
`v1.19`, and `v1.20`
- keep shared base logic in `flink-common`
- move version-specific SPI exposure to each versioned module
- add version-specific integration test entry classes for each Flink minor
version
- adapt provider-specific behavior where the runtime or validation path
differs by Flink minor version
This PR also fixes provider-specific compatibility and validation gaps:
- JDBC `1.19` and `1.20` now use the newer core JDBC catalog factory path
- Iceberg REST tests are aligned with version-specific SQL behavior and
local auxiliary service access
- Hive shared IT resources are resolved from classpath so versioned test
modules can reuse them
- Hive `1.20` keeps a version-specific assertion for the different
missing-option exception path
### Why are the changes needed?
The previous Flink multi-version layout still relied too heavily on shared
catalog entry implementations.
That made future extension harder when provider behavior or Flink APIs drift
by minor version. In practice, JDBC, Iceberg REST, and Hive validation paths
already showed real version-specific differences.
This PR makes each Flink minor version own its catalog/factory/IT entry
layer while still reusing shared base logic in `flink-common`. That keeps the
codebase closer to the Spark connector approach and makes future extensions,
including Flink `2.x`, easier to isolate and review.
Fix: #9710
### Does this PR introduce _any_ user-facing change?
Yes.
The produced Flink connector artifacts still target Flink `1.18`, `1.19`,
and `1.20`, but the internal implementation is now explicitly version-scoped:
- each supported Flink minor version has its own catalog/factory entry
classes
- each supported Flink minor version has its own integration-test entry
classes
This does not change the user-facing SQL syntax, but it changes how the
connector is organized and validated internally.
### How was this patch tested?
The following validations were run:
- `:flink-connector:flink-1.18:test -PskipITs`
- `:flink-connector:flink-1.19:test -PskipITs`
- `:flink-connector:flink-1.20:test -PskipITs`
Representative version-specific integration tests were also run:
- `GravitinoCatalogManagerIT118`
- `GravitinoCatalogManagerIT119`
- `GravitinoCatalogManagerIT120`
Provider-specific versioned integration tests were run for all supported
minors:
- Paimon filesystem:
- `FlinkPaimonLocalFileSystemBackendIT118`
- `FlinkPaimonLocalFileSystemBackendIT119`
- `FlinkPaimonLocalFileSystemBackendIT120`
- JDBC MySQL:
- `FlinkJdbcMysqlCatalogIT118`
- `FlinkJdbcMysqlCatalogIT119`
- `FlinkJdbcMysqlCatalogIT120`
- Iceberg REST:
- `FlinkIcebergRestCatalogIT118`
- `FlinkIcebergRestCatalogIT119`
- `FlinkIcebergRestCatalogIT120`
- Hive:
- `FlinkHiveCatalogIT118`
- `FlinkHiveCatalogIT119`
- `FlinkHiveCatalogIT120`
Additional test:
- `:integration-test-common:test --tests
"org.apache.gravitino.integration.test.util.TestBaseIT" -PskipITs`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
