[
https://issues.apache.org/jira/browse/KAFKA-1464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15123618#comment-15123618
]
Ismael Juma commented on KAFKA-1464:
------------------------------------
Thanks for your input [~jkreps].
With regards to the issue where a replica may never catch up, it is a good
point that came up previously. One option may be to disable throttling (or
increase the catch-up rate) in the case where the replica is falling further
behind.
One important question is whether users have enough information to be able to
configure an appropriate throttling/catch-up rate that takes into account both
disk IO and network bandwidth while keeping resource utilisation at an
appropriate level. Thoughts? (the log cleaner has a similar config:
`log.cleaner.io.max.bytes.per.second`, although it seems simpler to figure out).
> Add a throttling option to the Kafka replication tool
> -----------------------------------------------------
>
> Key: KAFKA-1464
> URL: https://issues.apache.org/jira/browse/KAFKA-1464
> Project: Kafka
> Issue Type: New Feature
> Components: replication
> Affects Versions: 0.8.0
> Reporter: mjuarez
> Assignee: Ismael Juma
> Priority: Minor
> Labels: replication, replication-tools
> Fix For: 0.9.1.0
>
>
> When performing replication on new nodes of a Kafka cluster, the replication
> process will use all available resources to replicate as fast as possible.
> This causes performance issues (mostly disk IO and sometimes network
> bandwidth) when doing this in a production environment, in which you're
> trying to serve downstream applications, at the same time you're performing
> maintenance on the Kafka cluster.
> An option to throttle the replication to a specific rate (in either MB/s or
> activities/second) would help production systems to better handle maintenance
> tasks while still serving downstream applications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
