[
https://issues.apache.org/jira/browse/FLINK-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15315316#comment-15315316
]
ASF GitHub Bot commented on FLINK-4018:
---------------------------------------
GitHub user tzulitai opened a pull request:
https://github.com/apache/flink/pull/2071
[FLINK-4018](streaming-connectors) Configurable idle time between
getRecords requests to Kinesis shards
Along with this new configuration and the already existing
`KinesisConfigConstants.CONFIG_SHARD_RECORDS_PER_GET`, users will have more
control on the desired throughput behaviour for the Kinesis consumer.
The default value for this new configuration is 500 milliseconds idle time.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tzulitai/flink FLINK-4018
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2071.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2071
----
commit 74fcb62f62007c0396d2ae870298f344445ae5ce
Author: Gordon Tai <gor...@vm5.com>
Date: 2016-06-04T04:43:30Z
[FLINK-4018] Add configuration for idle time between get requests to
Kinesis shards
----
> Configurable idle time between getRecords requests to Kinesis shards
> --------------------------------------------------------------------
>
> Key: FLINK-4018
> URL: https://issues.apache.org/jira/browse/FLINK-4018
> Project: Flink
> Issue Type: Sub-task
> Components: Streaming Connectors
> Affects Versions: 1.1.0
> Reporter: Tzu-Li (Gordon) Tai
> Assignee: Tzu-Li (Gordon) Tai
>
> Currently, the Kinesis consumer is calling getRecords() right after finishing
> previous calls. This results in easily reaching Amazon's limitation of 5 GET
> requests per shard per second. Although the code already has backoff & retry
> mechanism, this will affect other applications consuming from the same
> Kinesis stream.
> Along with this new configuration and the already existing
> `KinesisConfigConstants.CONFIG_SHARD_RECORDS_PER_GET`, users will have more
> control on the desired throughput behaviour for the Kinesis consumer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
