Sandeep, You need to have multiple replicas. Having single replica means you've one copy of the data and if that machine goes down there isn't another replica who can take over and be the leader for that partition.-Harsha
_____________________________ From: Sandeep Bishnoi <sandeepbishnoi.b...@gmail.com> Sent: Friday, June 19, 2015 2:37 PM Subject: Failure in Leader Election on broker shutdown To: <users@kafka.apache.org> Hi, I have a kafka cluster of three nodes. I have constructed a topic with the following command: bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 3 --topic testv1p3 So the topic "testv1p3" has 3 partitions and replication factor is 1. Here is the result of describe command: kafka_2.10-0.8.2.0]$ bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic testv1p3 Topic:testv1p3 PartitionCount:3 ReplicationFactor:1 Configs: Topic: testv1p3 Partition: 0 Leader: 1 Replicas: 1 Isr: 1 Topic: testv1p3 Partition: 1 Leader: 2 Replicas: 2 Isr: 2 Topic: testv1p3 Partition: 2 Leader: 0 Replicas: 0 Isr: 0 So far things are good. Now I tried to kill a broker using bin/kafka-server-stop.sh The broker was stopped successfully. Now I wanted to ensure that there is a new leader for the partition which was hosted on the terminated broker. Here is the output of describe command post broker termination: Topic:testv1p3 PartitionCount:3 ReplicationFactor:1 Configs: Topic: testv1p3 Partition: 0 Leader: 1 Replicas: 1 Isr: 1 Topic: testv1p3 Partition: 1 Leader: -1 Replicas: 2 Isr: Topic: testv1p3 Partition: 2 Leader: 0 Replicas: 0 Isr: 0 Leader for partition:1 is -1. Java API for kafka returns null for leader() in PartitionMetadata for partition 1. When I restarted the broker which was stopped earlier. Things go back to normal. 1) Does leader selection happen automatically ? 2) If yes, do I need any particular configuration in broker or topic config ? 3) If not, what is command to ensure that I have a leader for partition 1 in case its lead broker goes down. FYI I tried to run bin/bin/kafka-preferred-replica-election.sh --zookeeper localhost:2181 Post this script run, the topic description still remains same and no leader for partition 1. It will be great to get any help on this. Reference: Console log for (kafka-server-stop.sh): [2015-06-19 14:25:00,241] INFO [Kafka Server 2], shutting down (kafka.server.KafkaServer) [2015-06-19 14:25:00,243] INFO [Kafka Server 2], Starting controlled shutdown (kafka.server.KafkaServer) [2015-06-19 14:25:00,267] INFO [Kafka Server 2], Controlled shutdown succeeded (kafka.server.KafkaServer) [2015-06-19 14:25:00,273] INFO Deregistered broker 2 at path /brokers/ids/2. (kafka.utils.ZkUtils$) [2015-06-19 14:25:00,274] INFO [Socket Server on Broker 2], Shutting down (kafka.network.SocketServer) [2015-06-19 14:25:00,279] INFO [Socket Server on Broker 2], Shutdown completed (kafka.network.SocketServer) [2015-06-19 14:25:00,280] INFO [Kafka Request Handler on Broker 2], shutting down (kafka.server.KafkaRequestHandlerPool) [2015-06-19 14:25:00,282] INFO [Kafka Request Handler on Broker 2], shut down completely (kafka.server.KafkaRequestHandlerPool) [2015-06-19 14:25:00,600] INFO [Replica Manager on Broker 2]: Shut down (kafka.server.ReplicaManager) [2015-06-19 14:25:00,601] INFO [ReplicaFetcherManager on broker 2] shutting down (kafka.server.ReplicaFetcherManager) [2015-06-19 14:25:00,602] INFO [ReplicaFetcherManager on broker 2] shutdown completed (kafka.server.ReplicaFetcherManager) [2015-06-19 14:25:00,604] INFO [Replica Manager on Broker 2]: Shut down completely (kafka.server.ReplicaManager) [2015-06-19 14:25:00,605] INFO Shutting down. (kafka.log.LogManager) [2015-06-19 14:25:00,618] INFO Shutdown complete. (kafka.log.LogManager) [2015-06-19 14:25:00,620] WARN Kafka scheduler has not been started (kafka.utils.Utils$) java.lang.IllegalStateException: Kafka scheduler has not been started at kafka.utils.KafkaScheduler.ensureStarted(KafkaScheduler.scala:114) at kafka.utils.KafkaScheduler.shutdown(KafkaScheduler.scala:86) at kafka.controller.KafkaController.onControllerResignation(KafkaController.scala:350) at kafka.controller.KafkaController.shutdown(KafkaController.scala:664) at kafka.server.KafkaServer$$anonfun$shutdown$9.apply$mcV$sp(KafkaServer.scala:287) at kafka.utils.Utils$.swallow(Utils.scala:172) at kafka.utils.Logging$class.swallowWarn(Logging.scala:92) at kafka.utils.Utils$.swallowWarn(Utils.scala:45) at kafka.utils.Logging$class.swallow(Logging.scala:94) at kafka.utils.Utils$.swallow(Utils.scala:45) at kafka.server.KafkaServer.shutdown(KafkaServer.scala:287) at kafka.server.KafkaServerStartable.shutdown(KafkaServerStartable.scala:42) at kafka.Kafka$$anon$1.run(Kafka.scala:42) [2015-06-19 14:25:00,623] INFO Terminate ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread) [2015-06-19 14:25:00,625] INFO Session: 0x14de8e5f2b801f7 closed (org.apache.zookeeper.ZooKeeper) [2015-06-19 14:25:00,625] INFO EventThread shut down (org.apache.zookeeper.ClientCnxn) [2015-06-19 14:25:00,625] INFO [Kafka Server 2], shut down completed (kafka.server.KafkaServer) Regards, Sandeep