Do you see WARN messages like - "Error while fetching metadata [%s] for topic [%s]: %s " OR "Error while fetching metadata %s for topic partition [%s,%d]: [%s]"
On Wed, Oct 9, 2013 at 5:49 PM, Shafaq <s.abdullah...@gmail.com> wrote: > yes I'm using 0.8 head > > > roducer starting with option to multi-thread > ----compressed---true > Exception in thread "pool-1-thread-1" kafka.common.KafkaException: Failed > to fetch topic metadata for topic: 225topic1381362148396 > at > > kafka.producer.BrokerPartitionInfo.getBrokerPartitionInfo(BrokerPartitionInfo.scala:53) > at > > kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$getPartitionListForTopic(DefaultEventHandler.scala:185) > at > > kafka.producer.async.DefaultEventHandler$$anonfun$partitionAndCollate$1.apply(DefaultEventHandler.scala:149) > at > > kafka.producer.async.DefaultEventHandler$$anonfun$partitionAndCollate$1.apply(DefaultEventHandler.scala:148) > at > > scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:57) > at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:43) > at > > kafka.producer.async.DefaultEventHandler.partitionAndCollate(DefaultEventHandler.scala:148) > at > > kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:94) > at > > kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:72) > at kafka.producer.Producer.send(Producer.scala:74) > at kafka.javaapi.producer.Producer.send(Producer.scala:32) > at > > gree.relayx.producer.distributed.GreeKafkaProducerDistributed.run(GreeKafkaProducerDistributed.java:124) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:724) > > > ----Processing file multithreaded > mode----/mnt/prod_logs/relay_data_node-analytics-001_1380412084.gz with > compression enabled = true > Exception in thread "pool-1-thread-2" kafka.common.KafkaException: Failed > to fetch topic metadata for topic: 166topic1381362148385 > at > > kafka.producer.BrokerPartitionInfo.getBrokerPartitionInfo(BrokerPartitionInfo.scala:53) > at > > kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$getPartitionListForTopic(DefaultEventHandler.scala:185) > at > > kafka.producer.async.DefaultEventHandler$$anonfun$partitionAndCollate$1.apply(DefaultEventHandler.scala:149) > at > > kafka.producer.async.DefaultEventHandler$$anonfun$partitionAndCollate$1.apply(DefaultEventHandler.scala:148) > at > > scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:57) > at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:43) > at > > kafka.producer.async.DefaultEventHandler.partitionAndCollate(DefaultEventHandler.scala:148) > at > > kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:94) > at > > kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:72) > at kafka.producer.Producer.send(Producer.scala:74) > at kafka.javaapi.producer.Producer.send(Producer.scala:32) > at > > gree.relayx.producer.distributed.GreeKafkaProducerDistributed.run(GreeKafkaProducerDistributed.java:124) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:724) > > > On Wed, Oct 9, 2013 at 5:37 PM, Neha Narkhede <neha.narkh...@gmail.com > >wrote: > > > Are you using 0.8 HEAD ? > > > > Can you send around the full stack trace since one of the common reasons > > for failed topic metadata requests are socket timeouts. > > > > Thanks, > > Neha > > > > > > On Wed, Oct 9, 2013 at 4:30 PM, Shafaq <s.abdullah...@gmail.com> wrote: > > > > > I run into following error while doing some performance testing in > Kafka > > > with producers running in multiple threads. > > > > > > I can see the topic under /broker/topics in zooKeeper > > > > > > I believe the producer tries to get the metadata info from Zookeeper. > > > > > > I have tried to restart the 2-node Kafka broker cluster but of no use. > > > > > > Any idea what can cause this to happen? > > > I'm using kafka 0.8 version > > > > > > > > > --------------------------------------------------------------------------------------- > > > > > > Exception in thread "pool-1-thread-1" kafka.common.KafkaException: > Failed > > > to fetch topic metadata for topic: 37topic1381360774212 > > > at > > > > > > > > > kafka.producer.BrokerPartitionInfo.getBrokerPartitionInfo(BrokerPartitionInfo.scala:53) > > > at > > > > > > > > > kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$getPartitionListForTopic(DefaultEventHandler.scala:185) > > > -- > > > Kind Regards, > > > Shafaq > > > > > > > > > -- > Kind Regards, > Shafaq >
