Do you see any errors on the broker logs? Can you check the broker's
public access logs and see if there are topic metadata requests coming
in from the producer?
On Wed, Oct 29, 2014 at 07:15:15PM -0700, Rajiv Kurian wrote:
> I don't see anything else that is relevant. I traced the first of these
> error messages to figure out the ordering. It actually goes something like
> this:
>
> 2014-10-30T01:51:32.400Z ERROR [ProducerSendThread- ]
> [k.producer.async.DefaultEventHandler] {}: Failed to collate messages by
> topic, partition due to: null
>
> 2014-10-30T01:51:34.082Z ERROR [ProducerSendThread- ]
> [k.producer.async.DefaultEventHandler] {}: Failed to collate messages by
> topic, partition due to: null
>
> 2014-10-30T01:51:34.422Z ERROR [ProducerSendThread- ]
> [k.producer.async.DefaultEventHandler] {}: Failed to collate messages by
> topic, partition due to: null
>
> 2014-10-30T01:51:34.664Z ERROR [ProducerSendThread- ]
> [k.producer.async.DefaultEventHandler] {}: Failed to collate messages by
> topic, partition due to: null
>
> 2014-10-30T01:51:34.902Z ERROR [ProducerSendThread- ]
> [k.producer.async.DefaultEventHandler] {}: Failed to send requests for
> topics myTopic with correlation ids in [0,8]
>
> 2014-10-30T01:51:35.007Z ERROR [ProducerSendThread- ]
> [k.producer.async.ProducerSendThread ] {}: Error in handling batch of 1
> events
>
> kafka.common.FailedToSendMessageException: Failed to send messages after 3
> tries.
>
> at kafka.producer.async.DefaultEventHandler.handle(Unknown Source)
> ~[kafka_2.10-0.8.0.jar:0.8.0]
>
> at kafka.producer.async.ProducerSendThread.tryToHandle(Unknown Source)
> [kafka_2.10-0.8.0.jar:0.8.0]
>
> at
> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(Unknown
> Source) [kafka_2.10-0.8.0.jar:0.8.0]
>
> at
> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(Unknown
> Source) [kafka_2.10-0.8.0.jar:0.8.0]
>
> at scala.collection.immutable.Stream.foreach(Stream.scala:547)
> [scala-library-2.10.1.jar:na]
>
> at kafka.producer.async.ProducerSendThread.processEvents(Unknown Source)
> [kafka_2.10-0.8.0.jar:0.8.0]
>
> at kafka.producer.async.ProducerSendThread.run(Unknown Source)
> [kafka_2.10-0.8.0.jar:0.8.0]
>
> Thanks!
>
> On Wed, Oct 29, 2014 at 7:02 PM, Rajiv Kurian <ra...@signalfuse.com> wrote:
>
> > This pattern seems to repeat:
> >
> > 2014-10-30T01:54:46.004Z ERROR [ProducerSendThread- ]
> > [k.producer.async.DefaultEventHandler] {}: Failed to send requests for
> > topics myTopic with correlation ids in [1729,1736]
> >
> > 2014-10-30T01:54:46.008Z ERROR [ProducerSendThread- ]
> > [k.producer.async.ProducerSendThread ] {}: Error in handling batch of 4
> > events
> >
> > kafka.common.FailedToSendMessageException: Failed to send messages after 3
> > tries.
> >
> > at kafka.producer.async.DefaultEventHandler.handle(Unknown Source)
> > ~[kafka_2.10-0.8.0.jar:0.8.0]
> >
> > at kafka.producer.async.ProducerSendThread.tryToHandle(Unknown Source)
> > [kafka_2.10-0.8.0.jar:0.8.0]
> >
> > at
> > kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(Unknown
> > Source) [kafka_2.10-0.8.0.jar:0.8.0]
> >
> > at
> > kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(Unknown
> > Source) [kafka_2.10-0.8.0.jar:0.8.0]
> >
> > at scala.collection.immutable.Stream.foreach(Stream.scala:547)
> > [scala-library-2.10.1.jar:na]
> >
> > at kafka.producer.async.ProducerSendThread.processEvents(Unknown Source)
> > [kafka_2.10-0.8.0.jar:0.8.0]
> >
> > at kafka.producer.async.ProducerSendThread.run(Unknown Source)
> > [kafka_2.10-0.8.0.jar:0.8.0]
> >
> > 2014-10-30T01:54:46.025Z ERROR [ProducerSendThread- ]
> > [k.producer.async.DefaultEventHandler] {}: Failed to collate messages by
> > topic, partition due to: null
> >
> > 2014-10-30T01:54:46.174Z ERROR [ProducerSendThread- ]
> > [k.producer.async.DefaultEventHandler] {}: Failed to collate messages by
> > topic, partition due to: null
> >
> > 2014-10-30T01:54:46.356Z ERROR [ProducerSendThread- ]
> > [k.producer.async.DefaultEventHandler] {}: Failed to collate messages by
> > topic, partition due to: null
> >
> > 2014-10-30T01:54:46.644Z ERROR [ProducerSendThread- ]
> > [k.producer.async.DefaultEventHandler] {}: Failed to collate messages by
> > topic, partition due to: null
> >
> > On Wed, Oct 29, 2014 at 6:57 PM, Jun Rao <jun...@gmail.com> wrote:
> >
> >> The log before that will show you the cause of the error. Could you dig
> >> that out?
> >>
> >> Thanks,
> >>
> >> Jun
> >>
> >> On Wed, Oct 29, 2014 at 6:43 PM, Rajiv Kurian <ra...@signalfuse.com>
> >> wrote:
> >>
> >> > I keep seeing these errors in my code that is just trying to send some
> >> data
> >> > using an AsyncProducer:
> >> >
> >> > kafka.common.FailedToSendMessageException: Failed to send messages
> >> after 3
> >> > tries.
> >> >
> >> > at kafka.producer.async.DefaultEventHandler.handle(Unknown Source)
> >> > ~[kafka_2.10-0.8.0.jar:0.8.0]
> >> >
> >> > at kafka.producer.async.ProducerSendThread.tryToHandle(Unknown Source)
> >> > [kafka_2.10-0.8.0.jar:0.8.0]
> >> >
> >> > at
> >> >
> >> >
> >> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(Unknown
> >> > Source) [kafka_2.10-0.8.0.jar:0.8.0]
> >> >
> >> > at
> >> >
> >> >
> >> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(Unknown
> >> > Source) [kafka_2.10-0.8.0.jar:0.8.0]
> >> >
> >> > at scala.collection.immutable.Stream.foreach(Stream.scala:547)
> >> > [scala-library-2.10.1.jar:na]
> >> >
> >> > at kafka.producer.async.ProducerSendThread.processEvents(Unknown Source)
> >> > [kafka_2.10-0.8.0.jar:0.8.0]
> >> >
> >> > at kafka.producer.async.ProducerSendThread.run(Unknown Source)
> >> > [kafka_2.10-0.8.0.jar:0.8.0]
> >> >
> >> > 2014-10-30T01:40:45.176Z ERROR [ProducerSendThread- ]
> >> > [k.producer.async.DefaultEventHandler] {}: Failed to collate messages by
> >> > topic, partition due to: null
> >> >
> >> > 2014-10-30T01:40:45.506Z ERROR [ProducerSendThread- ]
> >> > [k.producer.async.DefaultEventHandler] {}: Failed to collate messages by
> >> > topic, partition due to: null
> >> >
> >> > 2014-10-30T01:40:45.647Z ERROR [ProducerSendThread- ]
> >> > [k.producer.async.DefaultEventHandler] {}: Failed to collate messages by
> >> > topic, partition due to: null
> >> >
> >> > 2014-10-30T01:40:45.772Z ERROR [ProducerSendThread- ]
> >> > [k.producer.async.DefaultEventHandler] {}: Failed to collate messages by
> >> > topic, partition due to: null
> >> >
> >> > 2014-10-30T01:40:45.890Z ERROR [ProducerSendThread- ]
> >> > [k.producer.async.DefaultEventHandler] {}: Failed to send requests for
> >> > topics myTopic with correlation ids in [169,176]
> >> >
> >> > 2014-10-30T01:40:45.892Z ERROR [ProducerSendThread- ]
> >> > [k.producer.async.ProducerSendThread ] {}: Error in handling batch of 29
> >> > events
> >> >
> >> >
> >> > I created the topic before starting using bin/kafka-topics.sh. I checked
> >> > zookeeper and seems like the topic was indeed created. Any ideas?
> >> >
> >>
> >
> >
