[
https://issues.apache.org/jira/browse/FLINK-40151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mao Jiayi updated FLINK-40151:
------------------------------
Description:
CDC YAML Transform supports SQL-like expressions, but some predicates and
nullable BOOLEAN logical expressions are not handled consistently with Flink
SQL.
Some predicates that are valid in Flink SQL are currently unsupported in YAML
Transform, such as IS UNKNOWN, IS DISTINCT FROM, LIKE ... ESCAPE, and SIMILAR
TO. In addition, logical expressions involving nullable BOOLEAN values may fail
at runtime when the operand is NULL, instead of producing the expected SQL
UNKNOWN result.
This issue focuses on:
- AND, OR, NOT with nullable BOOLEAN operands
- IS TRUE / IS NOT TRUE
- IS FALSE / IS NOT FALSE
- IS UNKNOWN / IS NOT UNKNOWN
- IS DISTINCT FROM / IS NOT DISTINCT FROM
- LIKE ... ESCAPE
- SIMILAR TO / NOT SIMILAR TO
UNKNOWN should be represented as null in projection results. In filter
expressions, only TRUE should pass; FALSE and UNKNOWN should not pass.
Changes to comparison operators, BETWEEN, IN, and the existing two-argument
LIKE / NOT LIKE behavior are out of scope for this issue and should be handled
separately.
was:
CDC YAML Transform currently does not fully align with Flink SQL semantics for
boolean expressions involving NULL.
Many comparison-related expressions return Java false when operands are NULL,
while SQL should return UNKNOWN. This affects comparison predicates, BETWEEN,
IN, and LIKE.
Examples:
{code:sql}
NULL = 1
NULL <> 1
NULL > 1
NULL BETWEEN 1 AND 3
1 IN (2, NULL)
NULL LIKE 'a%'
{code}
UNKNOWN should be represented as null in projection results. In filter
expressions, only TRUE should pass; FALSE and UNKNOWN should not pass.
This issue should align the following expressions with Flink SQL semantics:
* comparison operators: =, <>, <, <=, >, >=
* boolean operators needed for three-valued logic: AND, OR, NOT
* BETWEEN / NOT BETWEEN
* IN / NOT IN
* LIKE / NOT LIKE
* LIKE ... ESCAPE
* SIMILAR TO / NOT SIMILAR TO
* IS UNKNOWN / IS NOT UNKNOWN
* IS DISTINCT FROM / IS NOT DISTINCT FROM
CDC currently implements LIKE as Java regex matching. It should be changed to
SQL LIKE semantics:
* % matches zero or more characters
* _ matches one character
* ESCAPE is supported
* NULL operands return UNKNOWN
This issue is limited to NULL/UNKNOWN semantics for boolean predicates and
SQL-compatible LIKE/SIMILAR TO behavior. Other scalar function families should
be handled by separate issues.
Acceptance Criteria:
* NULL comparison results follow SQL three-valued logic.
* BETWEEN, IN, LIKE, and SIMILAR TO preserve UNKNOWN.
* LIKE no longer treats the pattern as Java regex.
* LIKE supports %, _, and ESCAPE according to SQL semantics.
* IS UNKNOWN, IS NOT UNKNOWN, IS DISTINCT FROM, and IS NOT DISTINCT FROM are
supported.
* YAML Transform specs cover the fixed NULL and LIKE/SIMILAR TO cases.
Summary: Support missing YAML Transform predicates and nullable BOOLEAN
logical evaluation (was: Align YAML Transform BETWEEN, IN, and LIKE semantics
with Flink SQL)
> Support missing YAML Transform predicates and nullable BOOLEAN logical
> evaluation
> ---------------------------------------------------------------------------------
>
> Key: FLINK-40151
> URL: https://issues.apache.org/jira/browse/FLINK-40151
> Project: Flink
> Issue Type: Improvement
> Components: Flink CDC
> Reporter: Mao Jiayi
> Assignee: Mao Jiayi
> Priority: Major
> Labels: pull-request-available
>
> CDC YAML Transform supports SQL-like expressions, but some predicates and
> nullable BOOLEAN logical expressions are not handled consistently with Flink
> SQL.
> Some predicates that are valid in Flink SQL are currently unsupported in YAML
> Transform, such as IS UNKNOWN, IS DISTINCT FROM, LIKE ... ESCAPE, and SIMILAR
> TO. In addition, logical expressions involving nullable BOOLEAN values may
> fail at runtime when the operand is NULL, instead of producing the expected
> SQL UNKNOWN result.
> This issue focuses on:
> - AND, OR, NOT with nullable BOOLEAN operands
> - IS TRUE / IS NOT TRUE
> - IS FALSE / IS NOT FALSE
> - IS UNKNOWN / IS NOT UNKNOWN
> - IS DISTINCT FROM / IS NOT DISTINCT FROM
> - LIKE ... ESCAPE
> - SIMILAR TO / NOT SIMILAR TO
> UNKNOWN should be represented as null in projection results. In filter
> expressions, only TRUE should pass; FALSE and UNKNOWN should not pass.
> Changes to comparison operators, BETWEEN, IN, and the existing two-argument
> LIKE / NOT LIKE behavior are out of scope for this issue and should be
> handled separately.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
