Hi Federico, What is your replica.lag.time.max.ms?
When acks=-1, the ProducerResponse won't return until all the broker in ISR get the message. During controlled shutdown, the shutting down broker is doing a lot of leader migration and could slow down on fetching data. The broker won't be kicked out of ISR until at least replica.lag.time.max.ms. Reducing the configuration will let the shutting down broker to be kicked out of ISR quicker if it cannot catch up. But if you set it too small, there could be more ISR expansion/shrinking. That said, currently controlled shutdown is not very efficient. We might improve it and hopefully later on it won't slow down the replication on the shutting down broker. Thanks, Jiangjie (Becket) Qin On Mon, Nov 2, 2015 at 5:52 AM, Federico Giraud <giraud.feder...@gmail.com> wrote: > Hi, > > I have few java async producers sending data to a 4-node Kafka cluster > version 0.8.2, containing few thousand topics, all with replication factor > 2. When i use acks=1 and trigger a controlled shutdown + restart on one > broker, the producers will send data to the new leader, reporting a very > low additional delay during the transition (as expected). However if i use > acks=-1, the producers will report a ~15 seconds delay between the send and > the future.get. Is this behavior expected? Is there a way to make it > faster? Or maybe it is a problem with my configuration? > > Broker configs: > broker.id=0 > log.dirs=/var/kafka/md1/kafka-logs > zookeeper.connect=10.40.27.107,10.40.27.108,10.40.27.109 > auto.create.topics.enable=true > default.replication.factor=2 > delete.topic.enable=true > log.retention.hours=24 > num.io.threads=5 > > Producer configs: > acks = -1 > retries = 3 > timeout.ms = 3000 > batch.size = 1048576 > linger.ms= 100 > metadata.fetch.timeout.ms = 5000 > metadata.max.age.ms = 60000 > > I tried different configurations, but i wasn't able to reduce the delay > during broker restart. The logs in the broker indicate that the controlled > shutdown was successful. > > Thank you > > regards, > Federico >
