Please check and ensure that each machine can recognize each other. It might be the case zookeepers and kafka are running seperately on each machine, but together they are not part of the same cluster. On 08-Aug-2015 6:34 am, "Scott Clasen" <sc...@heroku.com> wrote:
> each zk needs a myid file in the data dir, with a different number 1,2,3 > > http://zookeeper.apache.org/doc/r3.3.3/zookeeperAdmin.html > > You can find the meanings of these and other configuration settings in the > section Configuration Parameters > < > http://zookeeper.apache.org/doc/r3.3.3/zookeeperAdmin.html#sc_configuration > >. > A word though about a few here: > > Every machine that is part of the ZooKeeper ensemble should know about > every other machine in the ensemble. You accomplish this with the series of > lines of the form*server.id <http://server.id>=host:port:port*. The > parameters *host* and *port* are straightforward. You attribute the server > id to each machine by creating a file named myid, one for each server, > which resides in that server's data directory, as specified by the > configuration file parameter *dataDir*. > > On Fri, Aug 7, 2015 at 5:59 PM, Hemanth Abbina <heman...@eiqnetworks.com> > wrote: > > > Yes. I have set the unique broker ids (as 0, 1, 2) in the > > server.properties file > > > > I did not get the " set broker id in zookeeper data directory". We > don't > > set any broker id in zookeeper. > > > > We provide only "zookeeper.connect=ZK:2181,ZK2:2181,ZK3:2181" in > > server.properties file. Right ? > > > > -----Original Message----- > > From: Prabhjot Bharaj [mailto:prabhbha...@gmail.com] > > Sent: Friday, August 7, 2015 8:44 PM > > To: users@kafka.apache.org > > Subject: Re: Inconsistency with Zookeeper > > > > Have you set broker id in zookeeper data directory and some unique > broker > > Id in server.properties ? > > > > Regards, > > Prabhjot > > On Aug 6, 2015 1:43 PM, "Hemanth Abbina" <heman...@eiqnetworks.com> > wrote: > > > > > Hi, > > > > > > I am running a Kafka POC with below details > > > > > > * 3 Node cluster (4 Core, 16 GB RAM each) running Kafka > 0.8.2.1. > > > > > > * Each node running Kafka & Zookeeper instances. (So total of 3 > > > Kafka brokers & 3 zookeepers) > > > > > > When I tried to create a topic using the kafka-topics.sh, observing > > > the below exceptions on the console. If I try multiple times, the > > > topic is getting created at some time. Please suggest a workaround to > > > make this work every time. > > > > > > Thanks in advance. > > > > > > [ec2-user@ip-10-20-0-196 kafka_2.10-0.8.2.1]$ ./bin/kafka-topics.sh > > > --zookeeper 10.20.0.196:2181,10.20.0.197:2181,10.20.0.198:2181 --topic > > > reportspoc4 --create --partitions 1 --replication-factor 2 Error while > > > executing topic command replication factor: 2 larger than available > > > brokers: 1 > > > kafka.admin.AdminOperationException: replication factor: 2 larger than > > > available brokers: 1 > > > at > > > kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:70) > > > at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:171) > > > at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:93) > > > at kafka.admin.TopicCommand$.main(TopicCommand.scala:55) > > > at kafka.admin.TopicCommand.main(TopicCommand.scala) > > > > > > > > > [ec2-user@ip-10-20-0-196 kafka_2.10-0.8.2.1]$ ./bin/kafka-topics.sh > > > --zookeeper 10.20.0.196:2181,10.20.0.197:2181,10.20.0.198:2181 --topic > > > reportspoc4 --create --partitions 1 --replication-factor 2 Error while > > > executing topic command > > > org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode > > > = NoNode for /brokers/ids > > > org.I0Itec.zkclient.exception.ZkNoNodeException: > > > org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode > > > = NoNode for /brokers/ids > > > at > > > org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:47) > > > at > > > org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:685) > > > at org.I0Itec.zkclient.ZkClient.getChildren(ZkClient.java:413) > > > at org.I0Itec.zkclient.ZkClient.getChildren(ZkClient.java:409) > > > at kafka.utils.ZkUtils$.getChildren(ZkUtils.scala:462) > > > at kafka.utils.ZkUtils$.getSortedBrokerList(ZkUtils.scala:78) > > > at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:170) > > > at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:93) > > > at kafka.admin.TopicCommand$.main(TopicCommand.scala:55) > > > at kafka.admin.TopicCommand.main(TopicCommand.scala) > > > Caused by: org.apache.zookeeper.KeeperException$NoNodeException: > > > KeeperErrorCode = NoNode for /brokers/ids > > > at > > > org.apache.zookeeper.KeeperException.create(KeeperException.java:111) > > > at > > > org.apache.zookeeper.KeeperException.create(KeeperException.java:51) > > > at > > org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472) > > > at > > org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1500) > > > at > > > org.I0Itec.zkclient.ZkConnection.getChildren(ZkConnection.java:99) > > > at org.I0Itec.zkclient.ZkClient$2.call(ZkClient.java:416) > > > at org.I0Itec.zkclient.ZkClient$2.call(ZkClient.java:413) > > > at > > > org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675) > > > ... 8 more > > > > > > --regards > > > Hemanth > > > > > >
