lvyanquan commented on code in PR #4540: URL: https://github.com/apache/flink-cdc/pull/4540#discussion_r4044095706
########## docs/content/docs/core-concept/schema-evolution.md: ########## @@ -75,16 +75,27 @@ This is useful when your downstream sink is unready for any schema changes, but ## Existing Table Schema Expansion -Set the sink option `existing-table.schema-expansion.enabled` to `true` to try safe schema expansion when the initial `CreateTableEvent` encounters an existing target table. For sinks that implement this capability, the framework may add missing non-key physical columns as nullable columns and safely widen non-key column types. Derived DDL events are logged. +Set the sink option `existing-table.schema-expansion.mode` to control how the framework handles the initial `CreateTableEvent` when the target table already exists. The default is `OFF`. For sinks that implement this capability, the framework may add missing non-key physical columns as nullable columns and safely widen non-key column types. Derived DDL events are logged. -The default is `false`. Disabling this option does not disable the sink's own schema handling. The framework only derives DDL types that are enabled by `include.schema.changes` and supported by the sink; unsupported, unsafe, or failed expansions are delegated to the sink without introducing a framework fail-fast. +| Mode | Behavior on an existing target table | Behavior when the target table is missing | Failure handling | +|---|---|---|---| +| `OFF` | No check or expansion; the sink's original behavior applies | Sink creates the table | N/A | +| `CHECK` | Validate that every upstream column can be contained by the target table, without issuing any DDL | Fails the job; the table must be created externally | Any incompatibility, read failure, or missing capability fails the job with an aggregated error | +| `TRY_EXPAND` | Check and best-effort apply safe DDL, then verify the result by reading the target schema back | Sink creates the table | Failures of this mechanism are logged and delegated to the sink's original behavior | +| `EXPAND` | Check and apply safe DDL, then verify the result by reading the target schema back | Sink creates the table | Any incompatibility, unsupported DDL, execution or verification failure fails the job | + +`CHECK` never issues DDL, so it is independent of `include.schema.changes` and of the sink's DDL capabilities. It guards the initial table state and runs **regardless of `schema.change.behavior`** (including `IGNORE` and `EXCEPTION`); `TRY_EXPAND` and `EXPAND` skip the framework-side initial handling when `schema.change.behavior` is `IGNORE` or `EXCEPTION`. Note that `CHECK` only constrains the initial table handling: subsequent source schema changes are still controlled by `schema.change.behavior`, so it is not a job-wide "never issue DDL" switch. When the check fails, the aggregated error lists every difference (table, column, upstream type vs. target type) together with suggested `ALTER TABLE` repair statements that can be reviewed and adjusted to the target system's dialect. + +`TRY_EXPAND` swallows failures of this mechanism only; it neither hides errors from the sink's own schema handling nor guarantees that all upstream columns end up in the target table after a failed expansion. ```yaml sink: type: paimon - existing-table.schema-expansion.enabled: true + existing-table.schema-expansion.mode: "EXPAND" ``` +> Note: `existing-table.schema-expansion.enabled` is no longer supported. Use `existing-table.schema-expansion.mode` with one of `OFF`, `CHECK`, `TRY_EXPAND`, `EXPAND` instead; the previous `enabled: true` maps to `TRY_EXPAND`. Quote the mode value to avoid the bare `OFF` scalar being parsed as a YAML boolean. Review Comment: This note is unnecessary as existing-table.schema-expansion.enabled was never exposed to user. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
