[
https://issues.apache.org/jira/browse/KAFKA-1464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14005052#comment-14005052
]
Jon Bringhurst edited comment on KAFKA-1464 at 5/21/14 6:33 PM:
----------------------------------------------------------------
Although this would be nice to have, something similar exists as part of linux.
You can accomplish the same type of thing by first adding the process into a
net_cls cgroup. Then, you can use the tc command to classify the marked packets
into an htb qdisc (possibly with an sfq further down the tree to completely
prevent starvation) to throttle the packets coming from kafka.
* https://www.kernel.org/doc/Documentation/cgroups/net_cls.txt
* http://www.tldp.org/
* http://linux.die.net/man/8/tc
The blkio cgroup works in a similar way to throttle disk io.
was (Author: jonbringhurst):
Although this would be nice to have, something similar exists as part of linux.
You can accomplish the same type of thing by first adding the process into a
net_cls cgroup. Then, you can use the tc command to classify the marked packets
into an htb qdisc (possibly with an stb further down the tree to completely
prevent starvation) to throttle the packets coming from kafka.
* https://www.kernel.org/doc/Documentation/cgroups/net_cls.txt
* http://www.tldp.org/
* http://linux.die.net/man/8/tc
The blkio cgroup works in a similar way to throttle disk io.
> Add a throttling option to the Kafka replication tool
> -----------------------------------------------------
>
> Key: KAFKA-1464
> URL: https://issues.apache.org/jira/browse/KAFKA-1464
> Project: Kafka
> Issue Type: New Feature
> Components: replication
> Affects Versions: 0.8.0
> Reporter: Marcos Juarez
> Assignee: Neha Narkhede
> Priority: Minor
> Labels: replication, replication-tools
>
> When performing replication on new nodes of a Kafka cluster, the replication
> process will use all available resources to replicate as fast as possible.
> This causes performance issues (mostly disk IO and sometimes network
> bandwidth) when doing this in a production environment, in which you're
> trying to serve downstream applications, at the same time you're performing
> maintenance on the Kafka cluster.
> An option to throttle the replication to a specific rate (in either MB/s or
> activities/second) would help production systems to better handle maintenance
> tasks while still serving downstream applications.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
