Qilong Wang created FLINK-40303:
-----------------------------------
Summary: Sink constraint enforcer rejects key-only DELETE records
with NOT NULL non-key columns
Key: FLINK-40303
URL: https://issues.apache.org/jira/browse/FLINK-40303
Project: Flink
Issue Type: Bug
Components: Table SQL / Planner, Table SQL / Runtime
Affects Versions: 2.4.0
Reporter: Qilong Wang
Fix For: 2.4.0
### Problem
For an upsert pipeline where the source produces key-only DELETE records and
the sink accepts key-only DELETE records, ChangelogNormalize can be eliminated.
A key-only DELETE contains values only for primary key fields. All non-key
fields are set to null, regardless of their NOT NULL constraints. However, the
sink constraint enforcer currently validates every NOT NULL field and rejects
such records.
For example, given a sink schema with:
- `id INT PRIMARY KEY NOT ENFORCED`
- `name STRING NOT NULL`
the following valid key-only DELETE fails:
`-D[1, null]`
with an error stating that column `name` must not be null.
### Expected Behavior
For key-only DELETE records, NOT NULL constraints should only be enforced on
primary key fields. Non-key fields have no value semantics and should not be
validated.
INSERT, UPDATE, and full DELETE records should continue to enforce all NOT NULL
constraints.
### Root Cause
`CommonExecSink` did not pass the input changelog's key-only DELETE semantic or
the sink primary key indices to `ConstraintEnforcerExecutor`.
As a result, `NotNullConstraint` could not distinguish null placeholders in
key-only DELETE records from actual null values.
### Proposed Fix
- Pass `ChangelogMode.keyOnlyDeletes()` and the sink primary key indices from
`CommonExecSink` to `ConstraintEnforcerExecutor`.
- Let `NotNullConstraint` skip non-key fields when processing a key-only DELETE.
- Continue enforcing NOT NULL constraints on primary key fields and on all
fields for other changelog modes.
- Add a semantic test covering a key-only DELETE written to a sink with a NOT
NULL non-key column.
### Verification
Extend `DeletesByKeySemanticTests` with a source that emits `-D[1, null]` and a
delete-by-key sink whose non-key column is declared NOT NULL.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
