Zach, that error will occur if different brokers are advertising themselves in such a way where they are resolving to the same IP address. The advertised hostname is the hostname that will be given out to producers, consumers, and other brokers to connect to from the fetch meta data request.
Check what you have set as the advertised hostname value in server.properties for both of your brokers... If that is not set then something is wrong with your server/network configurations and DNS is resolving each broker's hostname to the same IP as a result of each box having to default to java.net.InetAddress.getCanonicalHostName() without that value set. /******************************************* Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> ********************************************/ On Tue, Jun 24, 2014 at 11:24 PM, Zack Payton <zpay...@gmail.com> wrote: > server.log has a lot of these errors: > [2014-06-24 20:07:16,124] ERROR [KafkaApi-6] error when handling request > Name: FetchRequest; Version: 0; CorrelationId: 81138; ClientId: > ReplicaFetche > rThread-0-5; ReplicaId: 6; MaxWait: 500 ms; MinBytes: 1 bytes; RequestInfo: > [test1,0] -> PartitionFetchInfo(0,1048576) (kafka.server.KafkaApis) > *kafka.common.KafkaException: Shouldn't set logEndOffset for replica 6 > partition [test1,0] since it's local* > at kafka.cluster.Replica.logEndOffset_$eq(Replica.scala:46) > at > > kafka.cluster.Partition.updateLeaderHWAndMaybeExpandIsr(Partition.scala:236) > at > > kafka.server.ReplicaManager.recordFollowerPosition(ReplicaManager.scala:432) > at > > kafka.server.KafkaApis$$anonfun$maybeUpdatePartitionHw$2.apply(KafkaApis.scala:460) > at > > kafka.server.KafkaApis$$anonfun$maybeUpdatePartitionHw$2.apply(KafkaApis.scala:458) > at scala.collection.immutable.Map$Map1.foreach(Map.scala:105) > at > kafka.server.KafkaApis.maybeUpdatePartitionHw(KafkaApis.scala:458) > at kafka.server.KafkaApis.handleFetchRequest(KafkaApis.scala:424) > at kafka.server.KafkaApis.handle(KafkaApis.scala:186) > at > kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:42) > at java.lang.Thread.run(Thread.java:701) > > > > On Tue, Jun 24, 2014 at 8:10 PM, Joe Stein <joe.st...@stealth.ly> wrote: > > > Are there any errors in the broker's logs? > > > > /******************************************* > > Joe Stein > > Founder, Principal Consultant > > Big Data Open Source Security LLC > > http://www.stealth.ly > > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > > ********************************************/ > > > > > > On Tue, Jun 24, 2014 at 9:56 PM, Zack Payton <zpay...@gmail.com> wrote: > > > > > Hi all, > > > > > > I have 3 zookeeper servers and 2 Kafka servers. > > > Running Kafka version 0.8.1.1. > > > Running zookeeper 3.3.5-cdh3u6. > > > From the Kafka servers I can access the zookeeper servers on 2181. > > > From one of the Kafka servers I can create a topic no problem: > > > [root@kafka1 kafka-0.8.1.1-src]# bin/kafka-topics.sh --zookeeper > > > zookeeper1:2181,zookeeper2:2181,zookeeper3 --create --topic test1 > > > --replication-factor 2 --partitions 2 > > > Created topic "test1". > > > > > > But, when I go to write to it I get the following: > > > [root@kafka1 kafka-0.8.1.1-src]# bin/kafka-console-producer.sh > > > --broker-list kafka1.queue.customer.sea1.ops.riotgames.com:9092, > > > kafka2.queue.customer.sea1.ops.riotgames.com:9092 --topic test1 > > > SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". > > > SLF4J: Defaulting to no-operation (NOP) logger implementation > > > SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for > > further > > > details. > > > TEST1 > > > [2014-06-24 18:49:15,753] WARN Error while fetching metadata > > > [{TopicMetadata for topic test1 -> > > > No partition metadata for topic test1 due to > > > kafka.common.LeaderNotAvailableException}] for topic [test1]: class > > > kafka.common.LeaderNotAvailableException > > > (kafka.producer.BrokerPartitionInfo) > > > [2014-06-24 18:49:15,813] WARN Error while fetching metadata partition > > > 0 leader: > > > none replicas: isr: isUnderReplicated: false for topic partition > > [test1,0]: > > > [class kafka.common.LeaderNotAvailableException] > > > (kafka.producer.BrokerPartitionInfo) > > > [2014-06-24 18:49:15,814] WARN Error while fetching metadata partition > > > 1 leader: > > > none replicas: isr: isUnderReplicated: false for topic partition > > [test1,1]: > > > [class kafka.common.LeaderNotAvailableException] > > > (kafka.producer.BrokerPartitionInfo) > > > [2014-06-24 18:49:15,821] WARN Failed to collate messages by > > > topic,partition due to: No leader for any partition in topic test1 > > > (kafka.producer.async.DefaultEventHandler) > > > [2014-06-24 18:49:15,933] WARN Error while fetching metadata > > > [{TopicMetadata for topic test1 -> > > > No partition metadata for topic test1 due to > > > kafka.common.LeaderNotAvailableException}] for topic [test1]: class > > > kafka.common.LeaderNotAvailableException > > > (kafka.producer.BrokerPartitionInfo) > > > [2014-06-24 18:49:15,933] WARN Failed to collate messages by > > > topic,partition due to: No leader for any partition in topic test1 > > > (kafka.producer.async.DefaultEventHandler) > > > [2014-06-24 18:49:16,041] WARN Error while fetching metadata > > > [{TopicMetadata for topic test1 -> > > > No partition metadata for topic test1 due to > > > kafka.common.LeaderNotAvailableException}] for topic [test1]: class > > > kafka.common.LeaderNotAvailableException > > > (kafka.producer.BrokerPartitionInfo) > > > [2014-06-24 18:49:16,042] WARN Failed to collate messages by > > > topic,partition due to: No leader for any partition in topic test1 > > > (kafka.producer.async.DefaultEventHandler) > > > [2014-06-24 18:49:16,148] WARN Error while fetching metadata partition > > > 0 leader: > > > none replicas: isr: isUnderReplicated: false for topic partition > > [test1,0]: > > > [class kafka.common.LeaderNotAvailableException] > > > (kafka.producer.BrokerPartitionInfo) > > > [2014-06-24 18:49:16,149] WARN Error while fetching metadata partition > > > 1 leader: > > > none replicas: isr: isUnderReplicated: false for topic partition > > [test1,1]: > > > [class kafka.common.LeaderNotAvailableException] > > > (kafka.producer.BrokerPartitionInfo) > > > [2014-06-24 18:49:16,149] WARN Failed to collate messages by > > > topic,partition due to: No leader for any partition in topic test1 > > > (kafka.producer.async.DefaultEventHandler) > > > [2014-06-24 18:49:16,257] WARN Error while fetching metadata > > > [{TopicMetadata for topic test1 -> > > > No partition metadata for topic test1 due to > > > kafka.common.LeaderNotAvailableException}] for topic [test1]: class > > > kafka.common.LeaderNotAvailableException > > > (kafka.producer.BrokerPartitionInfo) > > > [2014-06-24 18:49:16,260] ERROR Failed to send requests for topics > test1 > > > with correlation ids in [0,8] > (kafka.producer.async.DefaultEventHandler) > > > [2014-06-24 18:49:16,262] ERROR Error in handling batch of 1 events > > > (kafka.producer.async.ProducerSendThread) > > > kafka.common.FailedToSendMessageException: Failed to send messages > after > > 3 > > > tries. > > > at > > > > > > > > > kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90) > > > at > > > > > > > > > kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:104) > > > at > > > > > > > > > kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:87) > > > at > > > > > > > > > kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:67) > > > at scala.collection.immutable.Stream.foreach(Stream.scala:254) > > > at > > > > > > > > > kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:66) > > > at > > kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:44) > > > > > > > > > > > > Documentation on this issue is sparse. Any ideas? > > > > > > Thanks, > > > Z > > > > > >
