Could you try the 0.8.2.0 release? It fixed one issue related to topic creation.
Thanks, Jun On Wed, Feb 4, 2015 at 12:58 AM, Sumit Rangwala <sumitrangw...@gmail.com> wrote: > I am observing the following exception with kafka client: > > 2015-02-04 00:17:27,345 > > (LAX1-GRIFFIN-r8-1423037468055-pf13797-lax1-GriffinDownloader-1423037818264_c7b1e843ff51-1423037822122-eb7afca7-leader-finder-thread) > ClientUtils$ WARN: Fetching topic metadata with correlation id 112 for > topics [Set(LAX1-GRIFFIN-r8-1423037468055)] from broker > [id:49649,host:172.16.204.44,port:49649] failed > java.lang.ArrayIndexOutOfBoundsException: 7 > at > > kafka.api.TopicMetadata$$anonfun$readFrom$1.apply$mcVI$sp(TopicMetadata.scala:38) > at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:78) > at kafka.api.TopicMetadata$.readFrom(TopicMetadata.scala:36) > at > > kafka.api.TopicMetadataResponse$$anonfun$3.apply(TopicMetadataResponse.scala:31) > at > > kafka.api.TopicMetadataResponse$$anonfun$3.apply(TopicMetadataResponse.scala:31) > at > > scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233) > at > > scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233) > at scala.collection.immutable.Range.foreach(Range.scala:81) > at scala.collection.TraversableLike$class.map(TraversableLike.scala:233) > at scala.collection.immutable.Range.map(Range.scala:46) > at > kafka.api.TopicMetadataResponse$.readFrom(TopicMetadataResponse.scala:31) > at kafka.producer.SyncProducer.send(SyncProducer.scala:115) > at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58) > at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93) > at > > kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) > > > > # Background on what my code is doing: > > In my setup kafka brokers are set for auto topic creation. In the scenario > above a node informs other nodes (currently 5 in total) about ~50 new > (non-existent) topics, and all the nodes almost simultaneously open a > consumer for each of these topics. This triggers topic creation for all the > topics on kafka brokers. Most of the topics are created fine but there is > almost always few topics that throw the above exception and a kafka > producer is unable to send any data to such a topic > (LAX1-GRIFFIN-r8-1423037468055 > in the above case) > > > # Logs > All kafka broker logs (3 brokers) available at > http://d.pr/f/1eOGM/5UPMPfg5 > For these logs only LAX1-GRIFFIN-r8-1423037468055 had an issue. All other > topics were fine. > > > Setup > -------- > Zookeeper: 3.4.6 > Kafka broker: 0.8.2-beta > Kafka clients: 0.8.2-beta > > # Kafka boker settings (all other settings are default 0.8.2-beta settings) > kafka.controlled.shutdown.enable: 'FALSE' > kafka.auto.create.topics.enable: 'TRUE' > kafka.num.partitions: 8 > kafka.default.replication.factor: 1 > kafka.rebalance.backoff.ms: 3000 > kafka.rebalance.max.retries: 10 > kafka.log.retention.minutes: 1200 > kafka.delete.topic.enable: 'TRUE' > > > > Sumit >
