[
https://issues.apache.org/jira/browse/FLINK-19898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223556#comment-17223556
]
Danny Cranmer commented on FLINK-19898:
---------------------------------------
[~tzulitai] as discussed, I will publish a PR soon.
> [Kinesis][EFO] Ignore ReadTimeoutException from SubcribeToShard retry policy
> ----------------------------------------------------------------------------
>
> Key: FLINK-19898
> URL: https://issues.apache.org/jira/browse/FLINK-19898
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kinesis
> Reporter: Danny Cranmer
> Priority: Major
> Fix For: 1.12.0
>
>
> *Background*
> The Flink Kinesis EFO consumer has a {{SubscribeToShard}} retry policy which
> will terminate the job after a given number of subsequent attempt failures.
> In high backpressure scenarios the Netty HTTP Client throws a
> {{ReadTimeoutException}} when the consumer takes longer than 30s to process a
> batch. If this happens (by default) 10 times in a row, the job will
> terminate. There is no need to terminate in this condition, and the restart
> results in the job falling further behind.
> *Scope*
> Exclude the {{ReadTimeoutException}} from the {{SubscribeToShard}} retry
> policy, such that that connector will gracefully reconnect once the consumer
> has processed the queued records.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
