[
https://issues.apache.org/jira/browse/FLINK-40924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18123643#comment-18123643
]
sepuri sai krishna commented on FLINK-40924:
--------------------------------------------
I would like to work on this. Could someone assign the ticket to me?
> table.exec.sink.upsert-materialize=NONE silently bypasses the
> require-on-conflict check
> ---------------------------------------------------------------------------------------
>
> Key: FLINK-40924
> URL: https://issues.apache.org/jira/browse/FLINK-40924
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
> Affects Versions: 2.3.0, 2.4.0
> Reporter: sepuri sai krishna
> Priority: Major
>
> {{table.exec.sink.require-on-conflict}} defaults to {{true}} and is
> documented to throw when the
> query's upsert key differs from the sink's primary key and no {{ON CONFLICT}}
> clause is given.
> Setting {{table.exec.sink.upsert-materialize}} to {{NONE}} suppresses that
> error, and the query
> plans with no materializer.
> h3. Reproducer
> The upsert key is the grouping key {{c}}, which is never written to the sink,
> so it cannot equal
> the primary key {{x}}.
> {code:sql}
> CREATE TABLE src (a INT, b BIGINT, c STRING)
> WITH ('connector' = 'datagen', 'number-of-rows' = '5');
> CREATE TABLE snk (x INT, y BIGINT, PRIMARY KEY (x) NOT ENFORCED)
> WITH ('connector' = 'blackhole');
> SET 'table.exec.sink.upsert-materialize' = 'NONE';
> INSERT INTO snk SELECT MAX(a), COUNT(*) FROM src GROUP BY c;
> {code}
> With {{require-on-conflict}} left at its default of {{true}}:
> || upsert-materialize || result ||
> | {{AUTO}} (default) | {{ValidationException}}: "The query has an upsert key
> that differs from the primary key of the sink table ... Please specify an ON
> CONFLICT clause" |
> | {{FORCE}} | plans, {{upsertMaterialize=[true]}} |
> | {{NONE}} | plans, {{upsertMaterialize=[false]}} |
> Only {{AUTO}} raises the documented error.
> h3. Why this looks unintended
> The two options document separate concerns, and neither documents this
> interaction.
> {{require-on-conflict}} documents exactly one way to turn itself off:
> {quote}
> Set this to false to restore the old behavior where no ON CONFLICT clause was
> required. Note that
> disabling this check may lead to non-deterministic results in certain
> streaming scenarios.
> {quote}
> {{upsert-materialize}} is described entirely in terms of the materialize
> operator and shuffle
> disorder, and says nothing about validation or {{ON CONFLICT}}.
> So a user who sets {{NONE}} has opted out of materialization, not out of the
> check. The
> {{FORCE}} case also skips the error but still materializes, so its result
> stays deterministic;
> {{NONE}} skips the error and does not materialize.
> h3. Affects
> {{master}} and {{release-2.3}}, where the relevant code is unchanged.
> Related to FLINK-40899, which reports the opposite direction on the same
> validation: there it
> fires too early and masks an NDU error. This one is the check not firing at
> all.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
