[
https://issues.apache.org/jira/browse/FLINK-40240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mao Jiayi updated FLINK-40240:
------------------------------
Description:
Flink CDC YAML Transform currently supports {{CAST}}, {{COALESCE}}, {{IF}}, and
{{CASE}}, but common Flink SQL functions {{TRY_CAST}}, {{IFNULL}}, and
{{NULLIF}} cannot be used consistently. {{TRY_CAST}} and {{IFNULL}} are
unavailable, while {{NULLIF}} can be parsed but does not have complete
validation and runtime support.
This prevents users from safely handling malformed source values with
{{TRY_CAST}} and requires verbose alternative expressions for common NULL
handling. It also creates an inconsistency between Flink SQL and YAML Transform
expressions.
h2. Expected Behavior
The three functions should be available in both projection and filter
expressions with Flink SQL-compatible behavior. {{TRY_CAST}} should return
{{NULL}} for malformed data on supported conversion paths without changing
existing {{CAST}} behavior, while unsupported conversion paths should still be
rejected. {{IFNULL}} and {{NULLIF}} should follow Flink SQL value, type
inference, and nullability semantics.
was:
Flink CDC YAML Transform currently supports REGEXP_REPLACE, but the other
regular expression functions commonly available in Flink SQL are missing.
This limits the ability to extract, locate, and count pattern matches in
row-level transform expressions. Users currently need to implement UDFs for
these operations.
The missing functions include:
* REGEXP_EXTRACT
* REGEXP_EXTRACT_ALL
* REGEXP_COUNT
* REGEXP_INSTR
* REGEXP_SUBSTR
YAML Transform should support these functions with behavior aligned with Flink
SQL, including Java regular expression syntax, capture group indexes, NULL
arguments, invalid patterns, and unmatched input.
REGEXP_EXTRACT_ALL should return ARRAY<STRING> and work correctly in YAML
Transform projections.
Summary: Support TRY_CAST, IFNULL and NULLIF in Flink CDC YAML
Transform (was: Support additional Flink SQL regular expression functions in
YAML Transform)
> Support TRY_CAST, IFNULL and NULLIF in Flink CDC YAML Transform
> ---------------------------------------------------------------
>
> Key: FLINK-40240
> URL: https://issues.apache.org/jira/browse/FLINK-40240
> Project: Flink
> Issue Type: Improvement
> Components: Flink CDC
> Reporter: Mao Jiayi
> Priority: Major
> Labels: pull-request-available
>
> Flink CDC YAML Transform currently supports {{CAST}}, {{COALESCE}}, {{IF}},
> and {{CASE}}, but common Flink SQL functions {{TRY_CAST}}, {{IFNULL}}, and
> {{NULLIF}} cannot be used consistently. {{TRY_CAST}} and {{IFNULL}} are
> unavailable, while {{NULLIF}} can be parsed but does not have complete
> validation and runtime support.
> This prevents users from safely handling malformed source values with
> {{TRY_CAST}} and requires verbose alternative expressions for common NULL
> handling. It also creates an inconsistency between Flink SQL and YAML
> Transform expressions.
> h2. Expected Behavior
> The three functions should be available in both projection and filter
> expressions with Flink SQL-compatible behavior. {{TRY_CAST}} should return
> {{NULL}} for malformed data on supported conversion paths without changing
> existing {{CAST}} behavior, while unsupported conversion paths should still
> be rejected. {{IFNULL}} and {{NULLIF}} should follow Flink SQL value, type
> inference, and nullability semantics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
