Martijn Visser created FLINK-40077:
--------------------------------------
Summary: CommonExecSinkITCase.testNullEnforcer fails
intermittently: lost NoMoreSplitsEvent failover masks the NOT NULL enforcer
cause
Key: FLINK-40077
URL: https://issues.apache.org/jira/browse/FLINK-40077
Project: Flink
Issue Type: Bug
Components: Table SQL / Planner
Affects Versions: 2.4.0
Reporter: Martijn Visser
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=76751&view=results
(leg: test_cron_adaptive_scheduler table)
{code}
[Any cause contains message 'Column 'b' is NOT NULL, however, a null value is
being written into it. ...']
Expecting any element of:
...
org.apache.flink.runtime.JobException: Recovery is suppressed by
NoRestartBackoffTimeStrategy
org.apache.flink.util.FlinkException: An OperatorEvent from an
OperatorCoordinator to a task was lost. Triggering task failover to ensure
consistency. Event: '[NoMoreSplitEvent]', targetTask: Source: T1 ->
ConstraintEnforcer -> T1: Writer (3/4) - execution #0
org.apache.flink.runtime.operators.coordination.TaskNotRunningException:
Task is not running, but in state FAILED
{code}
Commit 3196a998cf0. The test asserts the job failure cause chain contains the
NOT NULL enforcer message, but the reported chain ends in the
lost-OperatorEvent failover instead; the enforcer error is absent.
Root cause: the Source V2 test source (introduced by FLINK-32695) emits all
rows on the first {{pollNext}}, before the coordinator delivers
{{NoMoreSplitsEvent}}. The NOT NULL enforcer then fails the chained Source ->
ConstraintEnforcer -> Writer task while that event is still in flight. Delivery
to the already-FAILED task throws {{TaskNotRunningException}}; because
{{NoMoreSplitsEvent}} is not loss-tolerant, {{SubtaskGatewayImpl}} escalates it
to a lost-OperatorEvent failover, which under {{NoRestartBackoffTimeStrategy}}
becomes the reported terminal cause and masks the enforcer error. Surfaces
under CI load; does not reproduce on an idle machine.
Relation to FLINK-40011: that ticket fixed the FINISHED-task variant in the
same class (readers returning END_OF_INPUT before the event arrived) by gating
completion on {{noMoreSplits}}. This is the FAILED-task variant, which the
completion-only gate cannot cover.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
