jiangxt2 opened a new pull request, #13498: URL: https://github.com/apache/gravitino/pull/13498
### What changes were proposed in this pull request? - Read `Normal` and `Aggregate` projection definitions from ClickHouse `system.projections` and expose them through the immutable, ClickHouse-specific `clickhouse.projections` table property. - Detect the optional `settings` column through `system.columns`; preserve safely reconstructable settings when the server exposes them. ClickHouse 24.9 does not expose this column. - Validate and emit equivalent inline projection clauses during `createTable`, including definitions originally added with native `ALTER TABLE ... ADD PROJECTION`. - Limit round-trip support to ClickHouse 24.9+ and the non-replicated MergeTree-family engines currently supported by this catalog: `MergeTree`, `ReplacingMergeTree`, `SummingMergeTree`, `AggregatingMergeTree`, `CollapsingMergeTree`, `VersionedCollapsingMergeTree`, and `GraphiteMergeTree`. Replicated MergeTree engines are out of scope. - Keep projection metadata in a connector-specific property. This change does not add a shared Projection API, parse table DDL as a fallback, manage projection ALTER/materialization operations, or copy projection data. ### Why are the changes needed? The ClickHouse catalog currently drops projection definitions from loaded table metadata. Creating a table from that metadata therefore loses the source table's projections. This change preserves supported definitions through the `loadTable`/`createTable` round-trip while keeping the general ClickHouse catalog baseline at 24.8.14. Fix: #11973 ### Does this PR introduce _any_ user-facing change? Yes. It adds the immutable `clickhouse.projections` table property for projection metadata round-tripping. Projection round-trip requires a server that provides `system.projections` (ClickHouse 24.9 or later); ordinary catalog metadata loading on 24.8 remains supported. Tables without projections do not receive this property. ### How was this patch tested? - `./gradlew :catalogs-contrib:catalog-jdbc-clickhouse:spotlessCheck`: passed. - `./gradlew :catalogs-contrib:catalog-jdbc-clickhouse:test -PskipITs`: 141 tests passed, with no failures, errors, or skips. - `./gradlew :catalogs-contrib:catalog-jdbc-clickhouse:test --tests 'org.apache.gravitino.catalog.clickhouse.integration.test.CatalogClickHouseProjectionIT' -PskipDockerTests=false`: 2 tests passed against ClickHouse 24.9.1.3278 and 25.12.2.54. - `./gradlew :catalogs-contrib:catalog-jdbc-clickhouse:test --tests 'org.apache.gravitino.catalog.clickhouse.integration.test.CatalogClickHouseIT.testCreateAndLoadClickhouseTable' -PskipDockerTests=false`: 1 test passed against ClickHouse 24.8.14. - `./gradlew :catalogs-contrib:catalog-jdbc-clickhouse:test --tests 'org.apache.gravitino.catalog.clickhouse.integration.test.CatalogClickHouseClusterIT' -PskipDockerTests=false`: 20 tests passed. - `./gradlew rat`: passed. The 24.8 targeted test and cluster suite ran before the final projection-property validation hardening. Those later changes are limited to projection-property parsing and are not exercised by the 24.8 missing-system-table path or the cluster suite. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
