This is because the producer rely on its metadata refresh to find the new leader of the partitions when the old leader fails. So if you are lucky that the next refresh happens immediately after the old leader is bounced the producer will send immediately, and if you are unlucky that the previous refresh just happened before the old leader is bounced you need to wait more time.
You can control "metadata.max.age.ms" as described in https://kafka.apache.org/documentation#producerconfigs to reduce the gap, but remember that too frequent metadata refreshes will increase load on the brokers. Guozhang On Wed, Nov 16, 2016 at 5:08 PM, ZHU Hua B <hua.b....@alcatel-lucent.com> wrote: > Hi Guozhang, > > > Thanks for your reply! > > The producer I used is Kafka 0.8.0, then upgrade to 0.10.0.0. There is no > error log on the producer client. Finally, I found the message could send, > but it is strange that some time the message could send immediately after > leader broker switchover, but some time it need to wait some sec, do you > know the reason? Thanks! > > > > > > > > > > Best Regards > > Johnny > > > -----Original Message----- > From: Guozhang Wang [mailto:wangg...@gmail.com] > Sent: 2016年11月17日 3:48 > To: users@kafka.apache.org > Subject: Re: Producer send message fail after upgrade broker from 0.8.0 to > 0.10.0.0 > > Johnny, > > Seems like you are already following the suggested procedure for upgrade: > > https://kafka.apache.org/documentation#upgrade > > What versions of the producer are you using that had such issues? And are > there any exception or error logs on the producer client? > > Guozhang > > > On Tue, Nov 15, 2016 at 1:28 AM, ZHU Hua B <hua.b....@alcatel-lucent.com> > wrote: > > > Hi All, > > > > > > I have three zookeepers, two Kafka brokers and some clients (consumer > > and > > producer) with current version 0.8.0, according to Kafka document, I > > do a rolling upgrade to 0.10.0.0 following below procedure: > > > > > > 1) Upgrade zookeeper one by one (shut down, update the code with > > 0.10.0.0, and restart it) > > > > 2) Add following properties into server.properties file of brokers: > > > > > > inter.broker.protocol.version=0.8.0 > > > > log.message.format.version=0.8.0 > > > > > > > > 3) Upgrade the first broker (shut down the broker, update the code > > with 0.10.0.0, and restart it) > > > > 4) Upgrade the second broker > > > > 5) After all the brokers started, set > inter.broker.protocol.version=0.10.0.0 > > of all the brokers. > > > > 6) Restart the brokers one by one > > > > 7) Upgrade all the clients one by one > > > > 8) Set log.message.format.version =0.10.0.0 of all the brokers. > > > > 9) Restart the brokers one by one > > > > After restart the first broder (new version) then shut down the second > > broker (old version) by step 4, the producer could not send message > > successfully (could not find new message from kafka log), if there is > > any mistake of my procedure? Thanks! > > > > By the way, the message could be sent successfully under below > > situation, once I shutdown the second broker (old version), the message > could not send. > > > > 1) Both first broker (new version) and second broker (old version) > > are launched > > > > 2) Only shutdown first broker (new version) > > > > 3) Only shutdown second broker while both brokers have old verson > > > > > > > > > > > > Best Regards > > > > Johnny > > > > > > > -- > -- Guozhang > -- -- Guozhang
