As a newbie, just setup my first Kafka 3 node, each in its own host cluster and its own ZK. Everything went fine, I could see three brokers registered in all three ZK /brokers/ids until I created a topic with this exception:
~/kafka/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic kafkatest [2016-05-14 02:22:40,003] INFO Partition [kafkatest,0] on broker 1: No checkpointed highwatermark is found for partition [kafkatest,0] (kafka.cluster.Partion) [2016-05-14 02:22:40,007] INFO [ReplicaFetcherManager on broker 1] Removed fetcher for partitions [kafkatest,0] (kafka.server.ReplicaFetcherManager) [2016-05-14 02:22:40,010] INFO Truncating log kafkatest-0 to offset 0. (kafka.log.Log) [2016-05-14 02:22:40,035] INFO [ReplicaFetcherThread-0-2], Starting (kafka.server.ReplicaFetcherThread) [2016-05-14 02:22:40,045] INFO [ReplicaFetcherManager on broker 1] Added fetcher for partitions List([[kafkatest,0], initOffset 0 to broker BrokerEndPoin2,localhost,9094)] ) (kafka.server.ReplicaFetcherManager) [2016-05-14 02:22:40,050] WARN [ReplicaFetcherThread-0-2], Error in fetch kafka.server.ReplicaFetcherThread$FetchRequest@4983f43d. Possible cause: java.iIOException: Connection to Node(2, localhost, 9094) failed (kafka.server.ReplicaFetcherThread) [2016-05-14 02:22:42,054] WARN [ReplicaFetcherThread-0-2], Error in fetch kafka.server.ReplicaFetcherThread$FetchRequest@256e458c. Possible cause: java.iIOException: Connection to Node(2, localhost, 9094) failed (kafka.server.ReplicaFetcherThread) [2016-05-14 02:22:44,055] WARN [ReplicaFetcherThread-0-2], Error in fetch kafka.server.ReplicaFetcherThread$FetchRequest@724f83b4. Possible cause: java.i Q.) Obviously, it could not recognize localhost, 9094, so what are some best practices and where should I include IP addresses or resolvable hostnames below? Without creating a topic, how do I know that the Kafka cluster has recognized all three brokers and is stable? Configuration File:::::::::::::::::::::::::::::: listeners=PLAINTEXT://localhost:9095 # The port the socket server listens on # port unique 9093, 9094,... port=9095 # Hostname the broker will bind to. If not set, the server will bind to all interfaces #host.name=localhost # Hostname the broker will advertise to producers and consumers. If not set, it uses the # value for "host.name" if configured. Otherwise, it will use the value returned from # java.net.InetAddress.getCanonicalHostName(). #advertised.host.name=<hostname routable by clients> # The port to publish to ZooKeeper for clients to use. If this is not set, # it will publish the same port that the broker binds to. #advertised.port=<port accessible by clients> kafka@awo-p05-kafk03:~$ netstat -atun | grep LISTEN tcp 0 0 0.0.0.0:8139 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN tcp6 0 0 :::38303 :::* LISTEN tcp6 0 0 :::2181 :::* LISTEN tcp6 0 0 127.0.0.1:9095 :::* LISTEN <<<<<<<<<<< tcp6 0 0 :::2888 :::* LISTEN tcp6 0 0 :::3888 :::* LISTEN tcp6 0 0 :::33042 :::* LISTEN tcp6 0 0 :::22 :::* LISTEN -- Krishna
