[
https://issues.apache.org/jira/browse/KAFKA-1464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371776#comment-15371776
]
Ralph Weires commented on KAFKA-1464:
-------------------------------------
Thanks a lot for the input - so if I understand this right, the config setting
James proposed would not work for me if I only set this on a single node (i.e.
the node under maintenance) before starting it up again, correct? Otherwise,
that would have been the perfect solution for me. I wouldn't mind running the
node with the custom setting during recovery, and just restarting it again once
more in the end without the setting.
If this won't work, what would even happen if this setting is defined
differently on various nodes in the cluster? Anyhow, alternatively I'd still
even consider using that option along with a full cluster restart before (and
disabling with another cluster restart afterwards), since a maintenance
scenario as described happens every now and then for us, and currently really
causes us major hassle for many hours, every time.
Jun - I'm also not be sure if disabling leader balancing during catch up would
necessarily be a good idea in general - but having / allowing the possibility
to configure this some way would be a nice option to have IMO.
> Add a throttling option to the Kafka replication tool
> -----------------------------------------------------
>
> Key: KAFKA-1464
> URL: https://issues.apache.org/jira/browse/KAFKA-1464
> Project: Kafka
> Issue Type: New Feature
> Components: replication
> Affects Versions: 0.8.0
> Reporter: mjuarez
> Assignee: Ben Stopford
> Priority: Minor
> Labels: replication, replication-tools
> Fix For: 0.10.1.0
>
>
> When performing replication on new nodes of a Kafka cluster, the replication
> process will use all available resources to replicate as fast as possible.
> This causes performance issues (mostly disk IO and sometimes network
> bandwidth) when doing this in a production environment, in which you're
> trying to serve downstream applications, at the same time you're performing
> maintenance on the Kafka cluster.
> An option to throttle the replication to a specific rate (in either MB/s or
> activities/second) would help production systems to better handle maintenance
> tasks while still serving downstream applications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
