Gustavo de Morais created FLINK-40874:
-----------------------------------------
Summary: Mini-batch streaming join evaluates non-equi conditions
on key-only deletes of folded pairs
Key: FLINK-40874
URL: https://issues.apache.org/jira/browse/FLINK-40874
Project: Flink
Issue Type: Sub-task
Components: Table SQL / API
Affects Versions: 2.1.3, 2.2.1, 2.3.0
Reporter: Gustavo de Morais
In mini-batch mode, a key-only DELETE followed by an accumulation of the same
unique key within one bundle is processed as a suppressed pair. Both calls of
the pair use the partial delete row: once to retract the old matches, and once
to decide whether a null padding change cancels out. With a non-equi join
condition on non-key columns, this produces wrong matches and wrong null
padding.
Example, mini-batch on: {{{}L LEFT JOIN R ON L.k = R.k AND R.v <>
'unknown'{}}}, where R is an upsert input with key-only deletes.
{code:java}
bundle 1: +I L(k=1), +I R(id=1, k=1, v=AIR) -> +I[L, AIR]
bundle 2: -D R(id=1) (key only), +I R(id=1, k=1, v=unknown) -> nothing
expected: -D[L, AIR], +I[L, null] {code}
Expected: -D[L, AIR], +I[L, null]. Actual: the null padding change is
suppressed based on the partial row.Proposed fix: look up the stored record
once per pair, and use it for both the retraction and the pair's null padding
check.Related: FLINK-40858 (the same fix for the non-mini-batch path).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
