Depending on the version of Kafka you're at, "controlled.shutdown.enable" should be set to true. If that's true and you always shutdown the broker cleanly (kill -15, not kill -9) and there are more than 1 replicas available, you should not see LeaderNotAvailable exceptions. If you kill the broker (kill -9) then Kafka does not get a chance to move the leaders away from the broken being shut down and the leader re-election can take some time leading to many LeaderNotAvailable exceptions.
You can verify the replica availability as well as leader movement through the kafka-topics command before shutting down zookeeper. Thanks Neha On Thu, Feb 19, 2015 at 10:51 AM, Felix GV <fville...@linkedin.com.invalid> wrote: > I'm not 100% sure, but I think this happens when ZK ephemeral znodes have > not had time to expire properly. When Kafka shuts down gracefully, it > should clean up its ephemeral nodes immediately (presumably, but that is > also an assumption... maybe it does have a short-coming in its graceful > shutdown logic). If Kafka gets killed improperly and bounced back up right > away, it cannot assume leadership properly because the ephemeral znodes of > the previous run are still there in ZK. > > I imagine Kafka could have some logic to deal with that better when it > gets fast-bounced... Alternatively, you may just have to wait a bit before > restarting Kafka after killing it. > > If anyone knows better, please correct me if I'm wrong. > > -- > > Felix GV > Data Infrastructure Engineer > Distributed Data Systems > LinkedIn > > f...@linkedin.com > linkedin.com/in/felixgv > > ________________________________________ > From: Chinmay Soman [chinmay.cere...@gmail.com] > Sent: Thursday, February 19, 2015 10:44 AM > To: dev@samza.apache.org > Subject: Question on hello-samza (Kafka startup and shutdown) > > Sending to a wider audience to know if anyone is also seeing this issue. > > It seems Kafka gets in a weird state everytime I do bin/grid stop all (and > then start all). > > I keep getting a LeaderNotAvailable exception on the producer side. It > seems this happens everytime Kafka hasn't been shut down properly. This > issue goes away if I use the following sequence: > > * bin/grid stop kafka > * bin/grid stop zookeeper (after like 5 seconds). > > (and then start everything). > > Has anyone else seen this ? > > -- > Thanks and regards > > Chinmay Soman >
