you can use the kafka-list-topic.sh to find out if leader for particual
topic is available.-1 in leader column might indicate trouble.
On Fri, Apr 25, 2014 at 6:34 AM, Guozhang Wang <wangg...@gmail.com> wrote:
> Could you double check if the topic LOGFILE04 is already created on the
> servers?
>
> Guozhang
>
>
> On Thu, Apr 24, 2014 at 10:46 AM, Yashika Gupta <
> yashika.gu...@impetus.co.in
> > wrote:
>
> > Jun,
> >
> > The detailed logs are as follows:
> >
> > 24.04.2014 13:37:31812 INFO main kafka.producer.SyncProducer -
> > Disconnecting from localhost:9092
> > 24.04.2014 13:37:38612 WARN main kafka.producer.BrokerPartitionInfo -
> > Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
> > No partition metadata for topic LOGFILE04 due to
> > kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class
> > kafka.common.LeaderNotAvailableException
> > 24.04.2014 13:37:40712 INFO main kafka.client.ClientUtils$ - Fetching
> > metadata from broker id:0,host:localhost,port:9092 with correlation id 1
> > for 1 topic(s) Set(LOGFILE04)
> > 24.04.2014 13:37:41212 INFO main kafka.producer.SyncProducer - Connected
> > to localhost:9092 for producing
> > 24.04.2014 13:37:48812 INFO main kafka.producer.SyncProducer -
> > Disconnecting from localhost:9092
> > 24.04.2014 13:37:48912 WARN main kafka.producer.BrokerPartitionInfo -
> > Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
> > No partition metadata for topic LOGFILE04 due to
> > kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class
> > kafka.common.LeaderNotAvailableException
> > 24.04.2014 13:37:49012 ERROR main
> kafka.producer.async.DefaultEventHandler
> > - Failed to collate messages by topic, partition due to: Failed to fetch
> > topic metadata for topic: LOGFILE04
> >
> >
> > 24.04.2014 13:39:96513 WARN
> >
> ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0
> > kafka.consumer.ConsumerFetcherThread -
> >
> [ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0],
> > Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 4;
> ClientId:
> >
> produceLogLine2-ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0;
> > ReplicaId: -1; MaxWait: 60000 ms; MinBytes: 1 bytes; RequestInfo:
> > [LOGFILE04,0] -> PartitionFetchInfo(2,1048576)
> > java.net.SocketTimeoutException
> > at
> > sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:229)
> > at
> sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
> > at
> >
> java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)
> > at kafka.utils.Utils$.read(Unknown Source)
> > at kafka.network.BoundedByteBufferReceive.readFrom(Unknown
> Source)
> > at kafka.network.Receive$class.readCompletely(Unknown Source)
> > at kafka.network.BoundedByteBufferReceive.readCompletely(Unknown
> > Source)
> > at kafka.network.BlockingChannel.receive(Unknown Source)
> > at kafka.consumer.SimpleConsumer.liftedTree1$1(Unknown Source)
> > at
> >
> kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(Unknown
> > Source)
> > at
> >
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(Unknown
> > Source)
> > at
> >
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
> > Source)
> > at
> >
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
> > Source)
> > at kafka.metrics.KafkaTimer.time(Unknown Source)
> > at
> > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(Unknown
> Source)
> > at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
> > Source)
> > at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
> > Source)
> > at kafka.metrics.KafkaTimer.time(Unknown Source)
> > at kafka.consumer.SimpleConsumer.fetch(Unknown Source)
> > at kafka.server.AbstractFetcherThread.processFetchRequest(Unknown
> > Source)
> > at kafka.server.AbstractFetcherThread.doWork(Unknown Source)
> > at kafka.utils.ShutdownableThread.run(Unknown Source)
> >
> >
> > Regards,
> > Yashika
> > ________________________________________
> > From: Jun Rao <jun...@gmail.com>
> > Sent: Thursday, April 24, 2014 10:49 PM
> > To: users@kafka.apache.org
> > Subject: Re: Kafka Performance Tuning
> >
> > Before that error messge, the log should tell you the cause of the error.
> > Could you dig that out?
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Thu, Apr 24, 2014 at 10:12 AM, Yashika Gupta <
> > yashika.gu...@impetus.co.in
> > > wrote:
> >
> > > Hi,
> > >
> > > I am working on a POC where I have 1 Zookeeper and 2 Kafka Brokers on
> my
> > > local machine. I am running 8 sets of Kafka consumers and producers
> > running
> > > in parallel.
> > >
> > > Below are my configurations:
> > > Consumer Configs:
> > > zookeeper.session.timeout.ms=120000
> > > zookeeper.sync.time.ms=2000
> > > zookeeper.connection.timeout.ms=120000
> > > auto.commit.interval.ms=60000
> > > rebalance.backoff.ms=2000
> > > fetch.wait.max.ms=60000
> > > auto.offset.reset=smallest
> > >
> > > Producer configs:
> > > key.serializer.class=kafka.serializer.StringEncoder
> > > request.required.acks=-1
> > > message.send.max.retries=3
> > > request.timeout.ms=60000
> > >
> > > I have tried various more configuration changes but I am running into
> the
> > > same exception again and again.
> > >
> > > 17.04.2014 06:31:83216 ERROR pool-5-thread-1
> > > kafka.producer.async.DefaultEventHandler - Failed to collate messages
> by
> > > topic, partition due to: Failed to fetch topic metadata for topic:
> > TOPIC99
> > > 17.04.2014 06:31:85616 ERROR pool-4-thread-1
> > > kafka.producer.async.DefaultEventHandler - Failed to collate messages
> by
> > > topic, partition due to: Failed to fetch topic metadata for topic:
> > TOPIC99
> > >
> > > I am not able to get to the root cause of the issue.
> > > Appreciate your help.
> > >
> > > Regards,
> > > Yashika
> > >
> > > ________________________________
> > >
> > >
> > >
> > >
> > >
> > >
> > > NOTE: This message may contain information that is confidential,
> > > proprietary, privileged or otherwise protected by law. The message is
> > > intended solely for the named addressee. If received in error, please
> > > destroy and notify the sender. Any use of this email is prohibited when
> > > received in error. Impetus does not represent, warrant and/or
> guarantee,
> > > that the integrity of this communication has been maintained nor that
> the
> > > communication is free of errors, virus, interception or interference.
> > >
> >
> > ________________________________
> >
> >
> >
> >
> >
> >
> > NOTE: This message may contain information that is confidential,
> > proprietary, privileged or otherwise protected by law. The message is
> > intended solely for the named addressee. If received in error, please
> > destroy and notify the sender. Any use of this email is prohibited when
> > received in error. Impetus does not represent, warrant and/or guarantee,
> > that the integrity of this communication has been maintained nor that the
> > communication is free of errors, virus, interception or interference.
> >
>
>
>
> --
> -- Guozhang
>
