Xiaobing Fang created FLINK-40850:
-------------------------------------
Summary: Fluss sink rejects pre-created primary-key tables with
custom bucket keys
Key: FLINK-40850
URL: https://issues.apache.org/jira/browse/FLINK-40850
Project: Flink
Issue Type: Bug
Components: Flink CDC
Reporter: Xiaobing Fang
h2. Description
The Fluss sink rejects a pre-created table when its bucket keys differ from the
default inferred from the primary key, even when the source and sink tables
have identical definitions.
h2. Steps to reproduce
# Pre-create identical Fluss source and sink tables with:
## Columns: \{{id INT NOT NULL, tenant INT NOT NULL, marker STRING}}
## Primary key: \{{(id, tenant)}}
## Bucket keys: \{{(tenant, id)}}
## Bucket count: \{{2}}
# Start a Fluss-to-Fluss CDC pipeline without sink \{{bucket.key}} or
\{{bucket.num}} overrides.
The initial \{{CreateTableEvent}} fails with:
{code}
ValidationException:
New Fluss table's bucket keys : [id, tenant]
Current Fluss's bucket keys: [tenant, id]
{code}
The same problem occurs when both tables use a valid bucket-key subset, such as
\{{(tenant)}}.
h2. Root cause
{\{FlussConversions.toFlussTable()}} defaults unspecified bucket keys to
primary keys minus partition keys. \{{FlussMetaDataApplier.sanityCheck()}} then
compares this inferred list with the existing table's bucket keys using
\{{List.equals}}, treating a table-creation default as an existing-table
constraint.
h2. Expected behavior and proposed fix
Accept the compatible pre-created table and preserve its layout.
For existing primary-key tables without an explicit bucket-key override, accept
target bucket keys that are a subset of the inferred keys, regardless of order.
Preserve strict ordered comparison for explicit overrides and leave new-table
creation unchanged.
Both cases were reproduced through \{{FlussMetadataApplierTest}} on
\{{3.7-SNAPSHOT}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
