[
https://issues.apache.org/jira/browse/FLINK-36455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17887948#comment-17887948
]
Arvid Heise commented on FLINK-36455:
-------------------------------------
Yes, all sinks following the FLIP-143 API are susceptible to the issue. After
implementing this ticket, graceful final checkpoints will never leave
transactions open (unless the connector is buggy). For normal checkpoints, open
transactions can only occur iff the RPC message notifyCheckpointCompleted is
lost (which cannot happen for final checkpoints).
Note that the issue is only relevant when transactions fail to commit on first
try, so everything is an edge case.
> Sink should commit everything on notifyCheckpointCompleted
> ----------------------------------------------------------
>
> Key: FLINK-36455
> URL: https://issues.apache.org/jira/browse/FLINK-36455
> Project: Flink
> Issue Type: Bug
> Components: API / Core
> Reporter: Arvid Heise
> Assignee: Arvid Heise
> Priority: Major
> Fix For: 2.0-preview
>
>
> Currently, we retry committables at some time later until they eventually
> succeed.
> However, that violates the contract of notifyCheckpointCompleted which states
> that all side effect must be committed before returning the method. In
> particular, notifyCheckpointCompleted must fail if we cannot guarantee that
> all side effects are committed for final checkpoints. As soon as
> notifyCheckpointCompleted returns, the final checkpoint is deemed completed,
> which currently may mean that some transactions are still open.
> The solution is that all retries must happen in a close loop in
> notifyCheckpointCompleted.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
