[
https://issues.apache.org/jira/browse/FLINK-34023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17805279#comment-17805279
]
Brad Atcheson commented on FLINK-34023:
---------------------------------------
Yes. I'll look into it, post a proposed solution approach here, and wait for
agreement before starting to implement.
> Expose Kinesis client retry config in sink
> ------------------------------------------
>
> Key: FLINK-34023
> URL: https://issues.apache.org/jira/browse/FLINK-34023
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kinesis
> Reporter: Brad Atcheson
> Priority: Major
>
> The consumer side exposes client retry configuration like
> [flink.shard.getrecords.maxretries|https://nightlies.apache.org/flink/flink-docs-stable/api/java/org/apache/flink/streaming/connectors/kinesis/config/ConsumerConfigConstants.html#SHARD_GETRECORDS_RETRIES]
> but the producer side lacks similar config for PutRecords.
> The KinesisStreamsSinkWriter constructor calls
> {code}
> this.httpClient =
> AWSGeneralUtil.createAsyncHttpClient(kinesisClientProperties);
> this.kinesisClient = buildClient(kinesisClientProperties, this.httpClient);
> {code}
> But those methods only refer to these values (aside from
> endpoint/region/creds) in the kinesisClientProperties:
> * aws.http-client.max-concurrency
> * aws.http-client.read-timeout
> * aws.trust.all.certificates
> * aws.http.protocol.version
> Without control over retry, users can observe exceptions like {code}Request
> attempt 2 failure: Unable to execute HTTP request: connection timed out after
> 2000 ms: kinesis.us-west-2.amazonaws.com{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
