[
https://issues.apache.org/jira/browse/FLINK-10074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16617382#comment-16617382
]
ASF GitHub Bot commented on FLINK-10074:
----------------------------------------
tweise commented on issue #6567: [FLINK-10074] Allowable number of checkpoint
failures
URL: https://github.com/apache/flink/pull/6567#issuecomment-421971822
@yanghua @tillrohrmann I think a user would normally expect this count to
apply globally, but please also consider the case of an intermittent failure
(like S3 rate limit or storage backend unavailable for other reason). In a
large job that would cause potentially many subtasks to fail in parallel. While
this could be addressed by setting a corresponding very high threshold, it
would in turn mean a problem that is isolated to a single task would not hit
the threshold until much much later, leaving the job in flipflop status instead
of failing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Allowable number of checkpoint failures
> ----------------------------------------
>
> Key: FLINK-10074
> URL: https://issues.apache.org/jira/browse/FLINK-10074
> Project: Flink
> Issue Type: Improvement
> Components: State Backends, Checkpointing
> Reporter: Thomas Weise
> Assignee: vinoyang
> Priority: Major
> Labels: pull-request-available
>
> For intermittent checkpoint failures it is desirable to have a mechanism to
> avoid restarts. If, for example, a transient S3 error prevents checkpoint
> completion, the next checkpoint may very well succeed. The user may wish to
> not incur the expense of restart under such scenario and this could be
> expressed with a failure threshold (number of subsequent checkpoint
> failures), possibly combined with a list of exceptions to tolerate.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
