[
https://issues.apache.org/jira/browse/FLINK-23495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17393734#comment-17393734
]
Nico Kruber commented on FLINK-23495:
-------------------------------------
Messages are only acknowledged once the checkpoint is complete on all tasks of
the job. If there is any failure during processing or the checkpoint, they
won't be ack'd and will be re-delivered after the job is restarted. If you ack
them right after receiving them, you will not get these messages again upon
failure and that only seems acceptable without checkpoints (which means no
failure tolerance)
> [GCP PubSub] Make checkpoint optional for preview/staging mode
> --------------------------------------------------------------
>
> Key: FLINK-23495
> URL: https://issues.apache.org/jira/browse/FLINK-23495
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Google Cloud PubSub
> Affects Versions: 1.13.0, 1.13.1
> Reporter: Brachi Packter
> Priority: Major
> Labels: pull-request-available
>
> I'm using PubSub connector with Flink sql.
> The issue that I get all the time error that PubSub required checkpoints, My
> question is if I/you can submit a PR that adds a property that can configure
> PubSub to start without checkpoints, and we can describe that it is just for
> preview/staging mode (interactive sql, Jupiter..)
> Other connectors support starting without checkpoints.
> What will be the impact for this change? I tried it locally and it seems to
> work ok.
> That is the code that always fail the source if no checkpoint is configured,
> i want to add some condition here:
> {code:java}
> if (hasNoCheckpointingEnabled(getRuntimeContext())) {
> throw new IllegalArgumentException( "The PubSubSource REQUIRES Checkpointing
> to be enabled and " + "the checkpointing frequency must be MUCH lower than
> the PubSub timeout for it to retry a message.");
> }
>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
