[
https://issues.apache.org/jira/browse/FLINK-20629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tzu-Li (Gordon) Tai reassigned FLINK-20629:
-------------------------------------------
Assignee: Danny Cranmer
> [Kinesis][EFO] Migrate from DescribeStream to DescribeStreamSummary
> -------------------------------------------------------------------
>
> Key: FLINK-20629
> URL: https://issues.apache.org/jira/browse/FLINK-20629
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kinesis
> Affects Versions: 1.12.0
> Reporter: Danny Cranmer
> Assignee: Danny Cranmer
> Priority: Minor
> Labels: pull-request-available
> Fix For: 1.12.1
>
>
> *Background*
> The Kinesis EFO connector invokes {{DescribeStream}} during startup to
> acquire the stream ARN. This call also includes the shard information and has
> a TPS of 10. A similar service exists, {{DescribeStreamSummary}} that has a
> TPS of 20 and a lighter response payload size.
> During startup sources with high parallelism compete to call this service (in
> {{LAZY}} mode), resulting in backoff and retry. Essentially the startup time
> can grow by 1s for every 10 parallelism, due to the 10 TPS. Migrating to
> {{DescribeStreamSummary}} will improve startup time.
> *Scope*
> Migrate call to {{DescribeStream}} to use {{DescribeStreamSummary}} instead.
> *Note*
> I have targeted {{1.12.1}}, let me know if we should instead target {{1.13}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
