Mao Jiayi created FLINK-40647:
---------------------------------
Summary: Support best-effort schema expansion for existing sink
tables during table creation
Key: FLINK-40647
URL: https://issues.apache.org/jira/browse/FLINK-40647
Project: Flink
Issue Type: New Feature
Components: Flink CDC
Reporter: Mao Jiayi
When the initial {{CreateTableEvent}} encounters an existing target table,
missing target columns or narrower target types may cause some table sinks to
ignore input fields and silently lose data. Similar handling may otherwise need
to be implemented separately in each connector.
This issue proposes an opt-in, best-effort schema expansion capability:
* Add the sink-level option {{{}existing-table.schema-expansion.enabled{}}},
disabled by default.
* Extend {{MetadataApplier}} with optional support for querying the existing
target schema, normalizing target types, and declaring column-name case
sensitivity.
* Only run during the initial {{{}CreateTableEvent{}}}.
* Add missing non-key physical columns as nullable columns.
* Safely widen non-key column types when the operation is supported.
* Log derived DDL events and verify successful changes by reading back the
target schema.
* Do not introduce new fail-fast behavior. Unsupported, unsafe, or failed
expansions are delegated to the sink’s existing schema handling.
Full compatibility validation, key-column modification, and replacing
connector-specific schema handling are out of scope.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
