Kafka replication throttling on broker level

2019-04-05 Thread Rahul Agarwal (Flipkart Data Group)
Hi, We are using Kafka 1.0.2 and wanted to rate limit when a broker is replaced and it syncs up with other brokers in the cluster. I followed the below doc for the same: https://kafka.apache.org/10/documentation.html#rep-throttle According to this document, leader.replication.throttled.rate, foll

Re: Race condition with stream use of Global KTable

2019-04-05 Thread Guozhang Wang
I see. So back to your original question, yes there will be a race condition since the global ktable is updated with a separate thread other than the one which is reading from topic-2 and process the record and query the global ktable. What I can think of on top of my head, is to 1) materialize t

Kafka replication throttling on broker level

2019-04-05 Thread Rahul Agarwal (Flipkart Data Group)
Hi, We are using Kafka 1.0.2 and wanted to rate limit when a broker is replaced and it syncs up with other brokers in the cluster. I followed the below doc for the same: https://kafka.apache.org/10/documentation.html#rep-throttle According to this document, leader.replication.throttled.rate, foll