If those errors only show up transiently when brokers are started, then
it's normal. It takes a bit of time for metadata to be probably to the
newly started brokers. You may have to increase # retries to avoid the
failure in the producer.
Thanks,
Jun
On Sun, Feb 23, 2014 at 6:33 PM, Arjun <ar...@socialtwist.com> wrote:
> Hi,
> No i have 3 kafka brokers running, but in the same system. I tried with
> the replication factor of 1 but it gives the same result.
>
> Thanks
> Arjun Narasimha Kota
>
>
> On Monday 24 February 2014 04:21 AM, Jun Rao wrote:
>
>> With only one broker, do you really want to have replication factor 2?
>> Maybe you could retry with replication factor 1.
>>
>> Thanks,
>>
>> Jun
>>
>>
>>
>>
>> On Sat, Feb 22, 2014 at 12:54 AM, Arjun <ar...@socialtwist.com> wrote:
>>
>> Hi please find below the o/p of the list and the console producer. There
>>> are no errors in the state change log or in the controller log.
>>>
>>> One thing i found is this one happens only at starting time. I mean the
>>> first time, when i start kafka and try to insert the messages i see this
>>> error.
>>>
>>>
>>> arjunn@PRINHYLTPDL0061:~/Kafka/kafka_2.8.0-0.8.0$ bin/kafka-run-class.sh
>>> kafka.tools.ConsumerOffsetChecker --group group1 --zkconnect
>>> localhost:2181,localhost:2182,localhost:2183 --topic test
>>> Group Topic Pid Offset logSize
>>> Lag Owner
>>>
>>> arjunn@PRINHYLTPDL0061:~/Kafka/kafka_2.8.0-0.8.0$
>>> bin/kafka-console-producer.sh --broker-list
>>> localhost:9092,localhost:9093,localhost:9094
>>> --topic test
>>> THis ia s test message
>>> [2014-02-22 14:16:26,924] WARN Error while fetching metadata
>>> [{TopicMetadata for topic test ->
>>>
>>> No partition metadata for topic test due to kafka.common.
>>> LeaderNotAvailableException}]
>>> for topic [test]: class kafka.common.LeaderNotAvailableException
>>> (kafka.producer.BrokerPartitionInfo)
>>> [2014-02-22 14:16:27,120] WARN Error while fetching metadata
>>> [{TopicMetadata for topic test ->
>>>
>>> No partition metadata for topic test due to kafka.common.
>>> LeaderNotAvailableException}]
>>> for topic [test]: class kafka.common.LeaderNotAvailableException
>>> (kafka.producer.BrokerPartitionInfo)
>>> [2014-02-22 14:16:27,121] ERROR Failed to collate messages by topic,
>>> partition due to: Failed to fetch topic metadata for topic: test
>>> (kafka.producer.async.DefaultEventHandler)
>>> [2014-02-22 14:16:27,259] WARN Error while fetching metadata
>>> [{TopicMetadata for topic test ->
>>>
>>> No partition metadata for topic test due to kafka.common.
>>> LeaderNotAvailableException}]
>>> for topic [test]: class kafka.common.LeaderNotAvailableException
>>> (kafka.producer.BrokerPartitionInfo)
>>> [2014-02-22 14:16:27,293] WARN Error while fetching metadata
>>> [{TopicMetadata for topic test ->
>>>
>>> No partition metadata for topic test due to kafka.common.
>>> LeaderNotAvailableException}]
>>> for topic [test]: class kafka.common.LeaderNotAvailableException
>>> (kafka.producer.BrokerPartitionInfo)
>>> [2014-02-22 14:16:27,295] ERROR Failed to collate messages by topic,
>>> partition due to: Failed to fetch topic metadata for topic: test
>>> (kafka.producer.async.DefaultEventHandler)
>>> [2014-02-22 14:16:27,443] WARN Error while fetching metadata
>>> [{TopicMetadata for topic test ->
>>>
>>> No partition metadata for topic test due to kafka.common.
>>> LeaderNotAvailableException}]
>>> for topic [test]: class kafka.common.LeaderNotAvailableException
>>> (kafka.producer.BrokerPartitionInfo)
>>> [2014-02-22 14:16:27,499] WARN Error while fetching metadata
>>> [{TopicMetadata for topic test ->
>>>
>>> No partition metadata for topic test due to kafka.common.
>>> LeaderNotAvailableException}]
>>> for topic [test]: class kafka.common.LeaderNotAvailableException
>>> (kafka.producer.BrokerPartitionInfo)
>>> [2014-02-22 14:16:27,499] ERROR Failed to collate messages by topic,
>>> partition due to: Failed to fetch topic metadata for topic: test
>>> (kafka.producer.async.DefaultEventHandler)
>>> [2014-02-22 14:16:27,630] WARN Error while fetching metadata
>>> [{TopicMetadata for topic test ->
>>>
>>> No partition metadata for topic test due to kafka.common.
>>> LeaderNotAvailableException}]
>>> for topic [test]: class kafka.common.LeaderNotAvailableException
>>> (kafka.producer.BrokerPartitionInfo)
>>> [2014-02-22 14:16:27,693] WARN Error while fetching metadata
>>> [{TopicMetadata for topic test ->
>>>
>>> No partition metadata for topic test due to kafka.common.
>>> LeaderNotAvailableException}]
>>> for topic [test]: class kafka.common.LeaderNotAvailableException
>>> (kafka.producer.BrokerPartitionInfo)
>>> [2014-02-22 14:16:27,694] ERROR Failed to collate messages by topic,
>>> partition due to: Failed to fetch topic metadata for topic: test
>>> (kafka.producer.async.DefaultEventHandler)
>>> [2014-02-22 14:16:27,881] WARN Error while fetching metadata
>>> [{TopicMetadata for topic test ->
>>>
>>> No partition metadata for topic test due to kafka.common.
>>> LeaderNotAvailableException}]
>>> for topic [test]: class kafka.common.LeaderNotAvailableException
>>> (kafka.producer.BrokerPartitionInfo)
>>> [2014-02-22 14:16:27,884] ERROR Failed to send requests for topics test
>>> with correlation ids in [0,8] (kafka.producer.async.DefaultEventHandler)
>>> [2014-02-22 14:16:27,886] ERROR Error in handling batch of 1 events
>>> (kafka.producer.async.ProducerSendThread)
>>>
>>> kafka.common.FailedToSendMessageException: Failed to send messages after
>>> 3 tries.
>>> at kafka.producer.async.DefaultEventHandler.handle(
>>> DefaultEventHandler.scala:90)
>>> at kafka.producer.async.ProducerSendThread.tryToHandle(
>>> ProducerSendThread.scala:104)
>>> at kafka.producer.async.ProducerSendThread$$anonfun$
>>> processEvents$3.apply(ProducerSendThread.scala:87)
>>> at kafka.producer.async.ProducerSendThread$$anonfun$
>>> processEvents$3.apply(ProducerSendThread.scala:67)
>>> at scala.collection.immutable.Stream.foreach(Stream.scala:254)
>>> at kafka.producer.async.ProducerSendThread.processEvents(
>>> ProducerSendThread.scala:66)
>>> at kafka.producer.async.ProducerSendThread.run(
>>> ProducerSendThread.scala:44)
>>>
>>>
>>> arjunn@PRINHYLTPDL0061:~/Kafka/kafka_2.8.0-0.8.0$
>>> bin/kafka-list-topic.sh
>>> --zookeeper localhost:2181
>>> topic: test partition: 0 leader: 2 replicas: 2,0 isr: 2,0
>>> topic: test partition: 1 leader: 0 replicas: 0,1 isr: 0,1
>>> topic: test partition: 2 leader: 1 replicas: 1,2 isr: 1,2
>>> topic: test partition: 3 leader: 2 replicas: 2,1 isr: 2,1
>>> topic: test partition: 4 leader: 0 replicas: 0,2 isr: 0,2
>>> topic: test partition: 5 leader: 1 replicas: 1,0 isr: 1,0
>>> topic: test partition: 6 leader: 2 replicas: 2,0 isr: 2,0
>>> topic: test partition: 7 leader: 0 replicas: 0,1 isr: 0,1
>>> topic: test partition: 8 leader: 1 replicas: 1,2 isr: 1,2
>>> topic: test partition: 9 leader: 2 replicas: 2,1 isr: 2,1
>>> topic: test partition: 10 leader: 0 replicas: 0,2 isr: 0,2
>>> topic: test partition: 11 leader: 1 replicas: 1,0 isr: 1,0
>>>
>>>
>>> What i did:
>>> started 3 kafka processes in my system.The config is default config with
>>> just the ports and log dir's changed.There are 3 zookeeper servers
>>> running.
>>> If i start the console producer next time it will work fine.
>>> To reproduce what did was, stopped all the kafka and zookeeper process,
>>> deleted the log dirs of the kafka and the zookeeper and restated the
>>> processes.
>>>
>>>
>>> Thanks
>>> Arjun Narasimha Kota
>>>
>>>
>>>
>>> On Friday 21 February 2014 09:41 PM, Jun Rao wrote:
>>>
>>> Could you do a list topic and show the output? Also, any error in the
>>>> controller and state-change log?
>>>>
>>>> Thanks,
>>>>
>>>> Jun
>>>>
>>>>
>>>> On Fri, Feb 21, 2014 at 2:33 AM, Arjun <ar...@socialtwist.com> wrote:
>>>>
>>>> hi,
>>>>
>>>>> I am testing kafka 0.8 on my local machine. i have only one zookeeper
>>>>> and
>>>>> one kafka broker running.
>>>>>
>>>>> when i run the console producer i get this error:
>>>>>
>>>>> [2014-02-21 16:01:20,512] WARN Error while fetching metadata
>>>>> [{TopicMetadata for topic test ->
>>>>> No partition metadata for topic test due to kafka.common.
>>>>> LeaderNotAvailableException}]
>>>>> for topic [test]: class kafka.common.LeaderNotAvailableException
>>>>> (kafka.producer.BrokerPartitionInfo)
>>>>> [2014-02-21 16:01:20,513] ERROR Failed to collate messages by topic,
>>>>> partition due to: Failed to fetch topic metadata for topic: test
>>>>> (kafka.producer.async.DefaultEventHandler)
>>>>> [2014-02-21 16:01:20,632] WARN Error while fetching metadata
>>>>> [{TopicMetadata for topic test ->
>>>>> No partition metadata for topic test due to kafka.common.
>>>>> LeaderNotAvailableException}]
>>>>> for topic [test]: class kafka.common.LeaderNotAvailableException
>>>>> (kafka.producer.BrokerPartitionInfo)
>>>>> [2014-02-21 16:01:20,636] ERROR Failed to send requests for topics test
>>>>> with correlation ids in [0,8] (kafka.producer.async.
>>>>> DefaultEventHandler)
>>>>> ERROR Error in handling batch of 1 events (kafka.producer.async.
>>>>> ProducerSendThread)
>>>>> kafka.common.FailedToSendMessageException: Failed to send messages
>>>>> after
>>>>> 3 tries.
>>>>> at kafka.producer.async.DefaultEventHandler.handle(
>>>>> DefaultEventHandler.scala:90)
>>>>> at kafka.producer.async.ProducerSendThread.tryToHandle(
>>>>> ProducerSendThread.scala:104)
>>>>> at kafka.producer.async.ProducerSendThread$$anonfun$
>>>>> processEvents$3.apply(ProducerSendThread.scala:87)
>>>>> at kafka.producer.async.ProducerSendThread$$anonfun$
>>>>> processEvents$3.apply(ProducerSendThread.scala:67)
>>>>> at scala.collection.immutable.Stream.foreach(Stream.scala:254)
>>>>> at kafka.producer.async.ProducerSendThread.processEvents(
>>>>> ProducerSendThread.scala:66)
>>>>> at kafka.producer.async.ProducerSendThread.run(
>>>>> ProducerSendThread.scala:44)
>>>>>
>>>>> Can some one please let me know why this is happening.
>>>>> I checked my vhost files and localhost, my hostname are correct.
>>>>>
>>>>>
>>>>>
>
