[
https://issues.apache.org/jira/browse/FLINK-40896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gustavo de Morais updated FLINK-40896:
--------------------------------------
Summary: FLIP-614: Add built-in DEDUPLICATE_KEEP_FIRST PTF (was: Add
built-in DEDUPLICATE_KEEP_FIRST PTF)
> FLIP-614: Add built-in DEDUPLICATE_KEEP_FIRST PTF
> -------------------------------------------------
>
> Key: FLINK-40896
> URL: https://issues.apache.org/jira/browse/FLINK-40896
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / API, Table SQL / Runtime
> Reporter: Gustavo de Morais
> Assignee: Vas Shabu
> Priority: Major
>
> Deduplication is a common Flink SQL need. Today users must express "keep
> first" dedup via verbose {{ROW_NUMBER()}} windowing, which must match an
> exact pattern to be planner-recognized and buries the intent in a windowed
> rank.
> This ticket adds {{{}DEDUPLICATE_KEEP_FIRST{}}}, a built-in PTF with two
> modes:
> * {*}Watermarkless{*}: first record per key wins, no watermark required.
> * {*}Event-time{*}: with {{{}on_time{}}}, the smallest-event-time row per
> key wins, finalized once the watermark passes it.
>
> {code:java}
> DEDUPLICATE_KEEP_FIRST(
> input => TABLE(t) [PARTITION BY <cols>],
> on_time => DESCRIPTOR(<rowtime>),
> state_ttl => <interval>,
> reset_ttl_on_duplicate => <boolean>,
> uid => <string>
> ) {code}
> Event-time mode requires insert-only input (updating input + {{on_time}}
> rejected at planning time). Watermarkless mode accepts updating input,
> absorbing later changes into the kept row.
> See
> [FLIP-614|https://cwiki.apache.org/confluence/spaces/FLINK/pages/451975182/FLIP-614+Add+built-in+DEDUPLICATE_KEEP_FIRST+PTF]
> for full motivation, rejected alternatives, and compatibility plan.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
