Thanks Neha, Indeed, there are no replicas apparently.
$ bin/kafka-topics.sh --describe --zookeeper localhost:2181 Topic:events PartitionCount:2 ReplicationFactor:1 Configs: Topic: events Partition: 0 Leader: 0 Replicas: 0 Isr: 0 Topic: events Partition: 1 Leader: 0 Replicas: 0 Isr: 0 Topic:mytopic PartitionCount:2 ReplicationFactor:1 Configs: Topic: mytopic Partition: 0 Leader: 0 Replicas: 0 Isr: 0 Topic: mytopic Partition: 1 Leader: 0 Replicas: 0 Isr: 0 Topic:mytopic2 PartitionCount:2 ReplicationFactor:1 Configs: Topic: mytopic2 Partition: 0 Leader: 0 Replicas: 0 Isr: 0 Topic: mytopic 2 Partition: 1 Leader: 0 Replicas: 0 Isr: 0 Topic:retry PartitionCount:2 ReplicationFactor:1 Configs: Topic: retry Partition: 0 Leader: 0 Replicas: 0 Isr: 0 Topic: retry Partition: 1 Leader: 0 Replicas: 0 Isr: 0 Is there any way to recover that? On Mon, Aug 11, 2014 at 9:25 PM, Neha Narkhede <neha.narkh...@gmail.com> wrote: > [2014-08-11 19:12:45,321] ERROR Controller 0 epoch 3 initiated state change > for partition [mytopic,0] from OfflinePartition to OnlinePartition failed > (state.change.logger) > kafka.common. > NoReplicaOnlineException: No replica for partition [mytopic,0] > is alive. Live brokers are: [Set()], Assigned replicas are: [List(0)] > at > > kafka.controller.OfflinePartitionLeaderSelector.selectLeader(PartitionLeaderSelector.scala:61) > > This error in the state change log suggests no assigned replicas for topic > mytopic, partition 0. Could you send around the output of the > kafka-topics.sh tool for this topic? Also, I don't think the apache mailing > lists allow attachments, so you might have to send a pastebin or something. > > Thanks, > Neha > > > On Mon, Aug 11, 2014 at 5:08 PM, Ryan Williams <rwilli...@gmail.com> > wrote: > > > The broker appears to be running > > > > $ telnet kafka-server 9092 > > Trying... > > Connected to kafka-server > > Escape character is '^]'. > > > > I've attached today's server.log. There was a manual restart of kafka, > > which you'll notice, but that didn't fix the issue. > > > > Thanks for looking! > > > > > > > > > > > > > > > > > > On Mon, Aug 11, 2014 at 4:30 PM, Guozhang Wang <wangg...@gmail.com> > wrote: > > > >> Hi Ryan, > >> > >> Could you check if all of your brokers are still live and running? Also > >> could you check the server log in addition to the producer / > state-change > >> / > >> controller logs? > >> > >> Guozhang > >> > >> > >> On Mon, Aug 11, 2014 at 12:45 PM, Ryan Williams <rwilli...@gmail.com> > >> wrote: > >> > >> > I have a single broker test Kafka instance that was running fine on > >> Friday > >> > (basically out of the box configuration with 2 partitions), now I come > >> back > >> > on Monday and producers are unable to send messages. > >> > > >> > What else can i look at to debug, and prevent? > >> > > >> > I know how to recover by removing data directories for kafka and > >> zookeeper > >> > to start fresh. But, this isn't the first time this has happened, so > I > >> > would like to understand it better to feel more comfortable with > kafka. > >> > > >> > > >> > =================== > >> > Producer error (from console produce) > >> > =================== > >> > [2014-08-11 19:32:49,781] WARN Error while fetching metadata > >> > [{TopicMetadata for topic mytopic -> > >> > No partition metadata for topic mytopic due to > >> > kafka.common.LeaderNotAvailableException}] for topic [mytopic]: class > >> > kafka.common.LeaderNotAvailableException > >> > (kafka.producer.BrokerPartitionInfo) > >> > [2014-08-11 19:32:49,782] ERROR Failed to collate messages by topic, > >> > partition due to: Failed to fetch topic metadata for topic: mytopic > >> > (kafka.producer.async.DefaultEventHandler) > >> > > >> > =============== > >> > state-change.log > >> > =============== > >> > [2014-08-11 19:12:45,312] TRACE Controller 0 epoch 3 started leader > >> > election for partition [mytopic,0] (state.change.logger) > >> > [2014-08-11 19:12:45,321] ERROR Controller 0 epoch 3 initiated state > >> change > >> > for partition [mytopic,0] from OfflinePartition to OnlinePartition > >> failed > >> > (state.change.logger) > >> > kafka.common.NoReplicaOnlineException: No replica for partition > >> [mytopic,0] > >> > is alive. Live brokers are: [Set()], Assigned replicas are: [List(0)] > >> > at > >> > > >> > > >> > kafka.controller.OfflinePartitionLeaderSelector.selectLeader(PartitionLeaderSelector.scala:61) > >> > [2014-08-11 19:12:45,312] TRACE Controller 0 epoch 3 started leader > >> > election for partition [mytopic,1] (state.change.logger) > >> > [2014-08-11 19:12:45,321] ERROR Controller 0 epoch 3 initiated state > >> change > >> > for partition [mytopic,1] from OfflinePartition to OnlinePartition > >> failed > >> > (state.change.logger) > >> > kafka.common.NoReplicaOnlineException: No replica for partition > >> [mytopic,1] > >> > is alive. Live brokers are: [Set()], Assigned replicas are: [List(0)] > >> > at > >> > > >> > > >> > kafka.controller.OfflinePartitionLeaderSelector.selectLeader(PartitionLeaderSelector.scala:61) > >> > > >> > =============== > >> > controller.log > >> > =============== > >> > [2014-08-11 19:12:45,308] DEBUG [OfflinePartitionLeaderSelector]: No > >> broker > >> > in ISR is alive for [mytopic,1]. Pick the leader from the alive > assigned > >> > replicas: (kafka.controller.OfflinePartitionLeaderSelector) > >> > [2014-08-11 19:12:45,321] DEBUG [OfflinePartitionLeaderSelector]: No > >> broker > >> > in ISR is alive for [mytopic,0]. Pick the leader from the alive > assigned > >> > replicas: (kafka.controller.OfflinePartitionLeaderSelector) > >> > > >> > >> > >> > >> -- > >> -- Guozhang > >> > > > > >