Have you checked the documentation on quotas? https://kafka.apache.org/documentation/#design_quotas
This should enable you to do what you're after. __ Sam Pegler PRODUCTION ENGINEER T. +44(0) 07 562 867 486 <http://www.infectiousmedia.com/> 3-7 Herbal Hill / London / EC1R 5EJ www.infectiousmedia.com This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately, and do not disclose the contents to another person, use it for any purpose, or store, or copy the information in any medium. Please also destroy and delete the message from your computer. On 10 July 2017 at 19:27, Łukasz Mierzwa <l.mier...@gmail.com> wrote: > Hi, > > looking at docs I see that Kafka seems to support throttling of > consumer/replication traffic, but I can't find anything that would suggest > you can prioritize one traffic type over another. > The problem: if at some point consumers starts to be lagging they will > start consuming messages as fast as they can, if there's enough lag to > recover from and if there's enough affected consumers that traffic can > easily saturate network on the leader, which can in turn affect replicas of > the partitions on that leader. > Is there a way to avoid replicas falling out of sync with the leader under > such scenario? Is there a way to prefer replication traffic over consumer > traffic? Or is throttling the only way to achieve this? Throttling solves > this, but it requires setting limits that can change over time, so it's bit > more manual and requires more maintenance than priority based solution. > > -- > Łukasz Mierzwa >
