[
https://issues.apache.org/jira/browse/KAFKA-1464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371578#comment-15371578
]
James Cheng edited comment on KAFKA-1464 at 7/11/16 8:47 PM:
-------------------------------------------------------------
[~r.weires], [~kzakee], you might be able to control this a little by setting
auto.leader.rebalance.enable=false. If you it to false, then the broker would
come up but would not assume leadership for any partitions at all, unless
manually told to. You would then have to use the
kafka-preferred-replica-election.sh tool
[https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-1.PreferredReplicaLeaderElectionTool],
to allow it to assume leadership.
This would mean that you wouldn't have the problem you described. But the
downside is that you are now in charge of handling rebalancing on your own.
The auto.leader.rebalance.enable flag is not changeable during runtime, tho. I
think it is only read at startup time.
was (Author: wushujames):
[~r.weires], you might be able to control this a little by setting
auto.leader.rebalance.enable=false. If you it to false, then the broker would
come up but would not assume leadership for any partitions at all, unless
manually told to. You would then have to use the
kafka-preferred-replica-election.sh tool
[https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-1.PreferredReplicaLeaderElectionTool],
to allow it to assume leadership.
This would mean that you wouldn't have the problem you described. But the
downside is that you are now in charge of handling rebalancing on your own.
The auto.leader.rebalance.enable flag is not changeable during runtime, tho. I
think it is only read at startup time.
> Add a throttling option to the Kafka replication tool
> -----------------------------------------------------
>
> Key: KAFKA-1464
> URL: https://issues.apache.org/jira/browse/KAFKA-1464
> Project: Kafka
> Issue Type: New Feature
> Components: replication
> Affects Versions: 0.8.0
> Reporter: mjuarez
> Assignee: Ben Stopford
> Priority: Minor
> Labels: replication, replication-tools
> Fix For: 0.10.1.0
>
>
> When performing replication on new nodes of a Kafka cluster, the replication
> process will use all available resources to replicate as fast as possible.
> This causes performance issues (mostly disk IO and sometimes network
> bandwidth) when doing this in a production environment, in which you're
> trying to serve downstream applications, at the same time you're performing
> maintenance on the Kafka cluster.
> An option to throttle the replication to a specific rate (in either MB/s or
> activities/second) would help production systems to better handle maintenance
> tasks while still serving downstream applications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
