Hmm, anything special with that topic (e.g., did you run any admin operation on it)? Also, does bouncing the brokers fix the issue?
Thanks, Jun On Tue, Feb 10, 2015 at 1:31 PM, Sumit Rangwala <sumitrangw...@gmail.com> wrote: > On Thu, Feb 5, 2015 at 12:37 PM, Sumit Rangwala <sumitrangw...@gmail.com> > wrote: > > > > > > > On Wed, Feb 4, 2015 at 9:23 PM, Jun Rao <j...@confluent.io> wrote: > > > >> Could you try the 0.8.2.0 release? It fixed one issue related to topic > >> creation. > >> > > > Jun, > > If you need more info let me know. Seems like TopicMetadataResponse is > expecting more fields then what is present in the response. > > > Sumit > > > > > > > Jun, > > > > Tried with 0.8.2.0 and I still see the same error. > > > > I see the error given below almost incessantly on the client side for > > topic LAX1-GRIFFIN-r47-1423165897627. Things look fine one the broker > side: > > Topic:LAX1-GRIFFIN-r47-1423165897627 PartitionCount:8 ReplicationFactor:1 > > Configs: > > Topic: LAX1-GRIFFIN-r47-1423165897627 Partition: 0 Leader: 49817 > Replicas: > > 49817 Isr: 49817 > > Topic: LAX1-GRIFFIN-r47-1423165897627 Partition: 1 Leader: 49818 > Replicas: > > 49818 Isr: 49818 > > Topic: LAX1-GRIFFIN-r47-1423165897627 Partition: 2 Leader: 49814 > Replicas: > > 49814 Isr: 49814 > > Topic: LAX1-GRIFFIN-r47-1423165897627 Partition: 3 Leader: 49817 > Replicas: > > 49817 Isr: 49817 > > Topic: LAX1-GRIFFIN-r47-1423165897627 Partition: 4 Leader: 49818 > Replicas: > > 49818 Isr: 49818 > > Topic: LAX1-GRIFFIN-r47-1423165897627 Partition: 5 Leader: 49814 > Replicas: > > 49814 Isr: 49814 > > Topic: LAX1-GRIFFIN-r47-1423165897627 Partition: 6 Leader: 49817 > Replicas: > > 49817 Isr: 49817 > > Topic: LAX1-GRIFFIN-r47-1423165897627 Partition: 7 Leader: 49818 > Replicas: > > 49818 Isr: 49818 > > > > > > Broker logs at http://d.pr/f/1jLOf/5H8uWPqu in case you need them > > > > ERROR: > > 2015-02-05 12:02:03,764 > > > (LAX1-GRIFFIN-r47-1423165897627-pf6898-lax1-GriffinDownloader-1423166442395_f27c6501f3d5-1423166444213-d572da8f-leader-finder-thread) > > ClientUtils$ WARN: Fetching topic metadata with correlation id 152 for > > topics [Set(LAX1-GRIFFIN-r47-1423165897627)] from broker > > [id:49814,host:172.16.204.44,port:49814] failed > > java.lang.ArrayIndexOutOfBoundsException: 7 > > at > > > kafka.api.TopicMetadata$$anonfun$readFrom$1.apply$mcVI$sp(TopicMetadata.scala:38) > > at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:78) > > at kafka.api.TopicMetadata$.readFrom(TopicMetadata.scala:36) > > at > > > kafka.api.TopicMetadataResponse$$anonfun$3.apply(TopicMetadataResponse.scala:31) > > at > > > kafka.api.TopicMetadataResponse$$anonfun$3.apply(TopicMetadataResponse.scala:31) > > at > > > scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233) > > at > > > scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233) > > at scala.collection.immutable.Range.foreach(Range.scala:81) > > at > scala.collection.TraversableLike$class.map(TraversableLike.scala:233) > > at scala.collection.immutable.Range.map(Range.scala:46) > > at > > kafka.api.TopicMetadataResponse$.readFrom(TopicMetadataResponse.scala:31) > > at kafka.producer.SyncProducer.send(SyncProducer.scala:114) > > at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58) > > at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93) > > at > > > kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66) > > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) > > > > > > > > > > > > > >> > >> Thanks, > >> > >> Jun > >> > >> On Wed, Feb 4, 2015 at 12:58 AM, Sumit Rangwala < > sumitrangw...@gmail.com> > >> wrote: > >> > >> > I am observing the following exception with kafka client: > >> > > >> > 2015-02-04 00:17:27,345 > >> > > >> > > >> > (LAX1-GRIFFIN-r8-1423037468055-pf13797-lax1-GriffinDownloader-1423037818264_c7b1e843ff51-1423037822122-eb7afca7-leader-finder-thread) > >> > ClientUtils$ WARN: Fetching topic metadata with correlation id 112 for > >> > topics [Set(LAX1-GRIFFIN-r8-1423037468055)] from broker > >> > [id:49649,host:172.16.204.44,port:49649] failed > >> > java.lang.ArrayIndexOutOfBoundsException: 7 > >> > at > >> > > >> > > >> > kafka.api.TopicMetadata$$anonfun$readFrom$1.apply$mcVI$sp(TopicMetadata.scala:38) > >> > at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:78) > >> > at kafka.api.TopicMetadata$.readFrom(TopicMetadata.scala:36) > >> > at > >> > > >> > > >> > kafka.api.TopicMetadataResponse$$anonfun$3.apply(TopicMetadataResponse.scala:31) > >> > at > >> > > >> > > >> > kafka.api.TopicMetadataResponse$$anonfun$3.apply(TopicMetadataResponse.scala:31) > >> > at > >> > > >> > > >> > scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233) > >> > at > >> > > >> > > >> > scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233) > >> > at scala.collection.immutable.Range.foreach(Range.scala:81) > >> > at > scala.collection.TraversableLike$class.map(TraversableLike.scala:233) > >> > at scala.collection.immutable.Range.map(Range.scala:46) > >> > at > >> > > >> > kafka.api.TopicMetadataResponse$.readFrom(TopicMetadataResponse.scala:31) > >> > at kafka.producer.SyncProducer.send(SyncProducer.scala:115) > >> > at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58) > >> > at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93) > >> > at > >> > > >> > > >> > kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66) > >> > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) > >> > > >> > > >> > > >> > # Background on what my code is doing: > >> > > >> > In my setup kafka brokers are set for auto topic creation. In the > >> scenario > >> > above a node informs other nodes (currently 5 in total) about ~50 new > >> > (non-existent) topics, and all the nodes almost simultaneously open a > >> > consumer for each of these topics. This triggers topic creation for > all > >> the > >> > topics on kafka brokers. Most of the topics are created fine but there > >> is > >> > almost always few topics that throw the above exception and a kafka > >> > producer is unable to send any data to such a topic > >> > (LAX1-GRIFFIN-r8-1423037468055 > >> > in the above case) > >> > > >> > > >> > # Logs > >> > All kafka broker logs (3 brokers) available at > >> > http://d.pr/f/1eOGM/5UPMPfg5 > >> > For these logs only LAX1-GRIFFIN-r8-1423037468055 had an issue. All > >> other > >> > topics were fine. > >> > > >> > > >> > Setup > >> > -------- > >> > Zookeeper: 3.4.6 > >> > Kafka broker: 0.8.2-beta > >> > Kafka clients: 0.8.2-beta > >> > > >> > # Kafka boker settings (all other settings are default 0.8.2-beta > >> settings) > >> > kafka.controlled.shutdown.enable: 'FALSE' > >> > kafka.auto.create.topics.enable: 'TRUE' > >> > kafka.num.partitions: 8 > >> > kafka.default.replication.factor: 1 > >> > kafka.rebalance.backoff.ms: 3000 > >> > kafka.rebalance.max.retries: 10 > >> > kafka.log.retention.minutes: 1200 > >> > kafka.delete.topic.enable: 'TRUE' > >> > > >> > > >> > > >> > Sumit > >> > > >> > > > > >
