Re: Requires suggestions for Producer request throttling

2017-06-30 Thread suku march
Hi hans, Thanks for your response. We are already looked this KIP-124 - Request rate quotas in Kafka 0.11.0, But it will help us to throttle based on request processing time taken by the user/client or broker CPU utilization of request thread. We are looking for some mechanism, like which wil

Re: Requires suggestions for Producer request throttling

2017-06-29 Thread Hans Jespersen
Request quotas was just added to 0.11. Does that help in your use case? https://cwiki.apache.org/confluence/display/KAFKA/KIP-124+-+Request+rate+quotas -hans > On Jun 29, 2017, at 12:55 AM, sukumar.np wrote: > > Hi Team, > > > > We are having a Kafka cluster with multiple Topics in it and s

Requires suggestions for Producer request throttling

2017-06-29 Thread sukumar.np
Hi Team, We are having a Kafka cluster with multiple Topics in it and shared with multiple services(clients). Each service will have multiple events source from where they will be pushing messages to Kafka brokers. Once a service starts producing message at a high rate, it will affect other s