each zk needs a myid file in the data dir, with a different number 1,2,3 http://zookeeper.apache.org/doc/r3.3.3/zookeeperAdmin.html
You can find the meanings of these and other configuration settings in the section Configuration Parameters <http://zookeeper.apache.org/doc/r3.3.3/zookeeperAdmin.html#sc_configuration>. A word though about a few here: Every machine that is part of the ZooKeeper ensemble should know about every other machine in the ensemble. You accomplish this with the series of lines of the form*server.id <http://server.id>=host:port:port*. The parameters *host* and *port* are straightforward. You attribute the server id to each machine by creating a file named myid, one for each server, which resides in that server's data directory, as specified by the configuration file parameter *dataDir*. On Fri, Aug 7, 2015 at 5:59 PM, Hemanth Abbina <heman...@eiqnetworks.com> wrote: > Yes. I have set the unique broker ids (as 0, 1, 2) in the > server.properties file > > I did not get the " set broker id in zookeeper data directory". We don't > set any broker id in zookeeper. > > We provide only "zookeeper.connect=ZK:2181,ZK2:2181,ZK3:2181" in > server.properties file. Right ? > > -----Original Message----- > From: Prabhjot Bharaj [mailto:prabhbha...@gmail.com] > Sent: Friday, August 7, 2015 8:44 PM > To: users@kafka.apache.org > Subject: Re: Inconsistency with Zookeeper > > Have you set broker id in zookeeper data directory and some unique broker > Id in server.properties ? > > Regards, > Prabhjot > On Aug 6, 2015 1:43 PM, "Hemanth Abbina" <heman...@eiqnetworks.com> wrote: > > > Hi, > > > > I am running a Kafka POC with below details > > > > * 3 Node cluster (4 Core, 16 GB RAM each) running Kafka 0.8.2.1. > > > > * Each node running Kafka & Zookeeper instances. (So total of 3 > > Kafka brokers & 3 zookeepers) > > > > When I tried to create a topic using the kafka-topics.sh, observing > > the below exceptions on the console. If I try multiple times, the > > topic is getting created at some time. Please suggest a workaround to > > make this work every time. > > > > Thanks in advance. > > > > [ec2-user@ip-10-20-0-196 kafka_2.10-0.8.2.1]$ ./bin/kafka-topics.sh > > --zookeeper 10.20.0.196:2181,10.20.0.197:2181,10.20.0.198:2181 --topic > > reportspoc4 --create --partitions 1 --replication-factor 2 Error while > > executing topic command replication factor: 2 larger than available > > brokers: 1 > > kafka.admin.AdminOperationException: replication factor: 2 larger than > > available brokers: 1 > > at > > kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:70) > > at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:171) > > at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:93) > > at kafka.admin.TopicCommand$.main(TopicCommand.scala:55) > > at kafka.admin.TopicCommand.main(TopicCommand.scala) > > > > > > [ec2-user@ip-10-20-0-196 kafka_2.10-0.8.2.1]$ ./bin/kafka-topics.sh > > --zookeeper 10.20.0.196:2181,10.20.0.197:2181,10.20.0.198:2181 --topic > > reportspoc4 --create --partitions 1 --replication-factor 2 Error while > > executing topic command > > org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode > > = NoNode for /brokers/ids > > org.I0Itec.zkclient.exception.ZkNoNodeException: > > org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode > > = NoNode for /brokers/ids > > at > > org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:47) > > at > > org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:685) > > at org.I0Itec.zkclient.ZkClient.getChildren(ZkClient.java:413) > > at org.I0Itec.zkclient.ZkClient.getChildren(ZkClient.java:409) > > at kafka.utils.ZkUtils$.getChildren(ZkUtils.scala:462) > > at kafka.utils.ZkUtils$.getSortedBrokerList(ZkUtils.scala:78) > > at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:170) > > at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:93) > > at kafka.admin.TopicCommand$.main(TopicCommand.scala:55) > > at kafka.admin.TopicCommand.main(TopicCommand.scala) > > Caused by: org.apache.zookeeper.KeeperException$NoNodeException: > > KeeperErrorCode = NoNode for /brokers/ids > > at > > org.apache.zookeeper.KeeperException.create(KeeperException.java:111) > > at > > org.apache.zookeeper.KeeperException.create(KeeperException.java:51) > > at > org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472) > > at > org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1500) > > at > > org.I0Itec.zkclient.ZkConnection.getChildren(ZkConnection.java:99) > > at org.I0Itec.zkclient.ZkClient$2.call(ZkClient.java:416) > > at org.I0Itec.zkclient.ZkClient$2.call(ZkClient.java:413) > > at > > org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675) > > ... 8 more > > > > --regards > > Hemanth > > >
