Whats the size of the batch that you are sending? If the messages produced are more that 4000 at any point of time there is a chance of the replica falling behind more than 4000 messages and being kicked out of the ISR. This happens because the thread that checks for in sync replicas is asynchronous. The replica gets added back once it catches up. We at Linkedin have seen this in our test clusters.
If this happens and you loose the leader broker at this point you will have unclean leader election. If this is the case you should increase the replica.max.lag.message parameter value. Thanks, Mayuresh On Thu, Mar 5, 2015 at 2:50 PM, xiaoyu wang <xiaoyu.w...@gmail.com> wrote: > Hi all, > > We previously have replica.max.lag.message set to 4000 and use sync > producer to send data to kafka, one message at a time. With this, we don't > see many unclean leader election. > > Recently, we switched to use sync producer and batch messages. After that, > we see unclean leader election more often. Did you observe the same? > > Do you have recommendations about how broker configuration should be > adjusted based on the batch size? > > Thanks, > -- -Regards, Mayuresh R. Gharat (862) 250-7125
