Martijn Visser created FLINK-40898:
--------------------------------------
Summary: TRY_RESOLVE wrongly rejects non-deterministic filters
inside ChangelogNormalize
Key: FLINK-40898
URL: https://issues.apache.org/jira/browse/FLINK-40898
Project: Flink
Issue Type: Bug
Components: Table SQL / Planner
Reporter: Martijn Visser
Since FLINK-39313 the NDU check rejects a non-deterministic condition that
{{PushCalcPastChangelogNormalizeRule}} pushed into ChangelogNormalize. The
operator applies that condition to inserts and updates only and emits
retractions from state
({{DeduplicateFunctionHelper#processLastRowOnChangelogWithFilter}}), so it
cannot drop a retraction and the rejection is a false positive. A probe with a
UDF that is true on its first call only ends with an empty sink here, while
every Calc placement keeps the stale row.
We should drop the check in {{visitChangelogNormalize}}, or downgrade it to a
note that the condition is evaluated once per key, and keep the Calc checks
from FLINK-28737. The upsert-kafka shape from FLINK-35341 has been handled by
this placement since FLINK-37390.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
