Hello David, This is a known issue with consumer starting before the topic is created:
https://issues.apache.org/jira/browse/KAFKA-1006 And we are still figuring the right way to fix it. Could you describe your use cases where you need to start the consumer beforehand? Guozhang On Thu, Mar 13, 2014 at 2:42 AM, David Morales de Frías < dmora...@paradigmatecnologico.com> wrote: > Hi there, > > A few days ago i have written about an issue (0.8.0) when auto-create topic > is enabled and consumer starts before producer (consumer never sees any > message and they seem to be lost even by the brokers) > > Now i have done the same test with 0.8.1 and the issue is partially fixed > but now the consumer never sees the first message in the topic. > > These are the steps: > > 1) kafka-console-consumer --topic newTopic (it doesn't exist) > 2) kafka-console-producer --topic newTopic > 3) Send the first messages > 4) I can see this in the log and the consumer never receives this first > message > > [2014-03-13 10:24:07,223] WARN Error while fetching metadata > [{TopicMetadata for topic test3 -> > No partition metadata for topic test3 due to > kafka.common.LeaderNotAvailableException}] for topic [test3]: class > kafka.common.LeaderNotAvailableException > (kafka.producer.BrokerPartitionInfo) > [2014-03-13 10:24:07,237] WARN Error while fetching metadata > [{TopicMetadata for topic test3 -> > No partition metadata for topic test3 due to > kafka.common.LeaderNotAvailableException}] for topic [test3]: class > kafka.common.LeaderNotAvailableException > (kafka.producer.BrokerPartitionInfo) > [2014-03-13 10:24:07,238] ERROR Failed to collate messages by topic, > partition due to: Failed to fetch topic metadata for topic: test3 > (kafka.producer.async.DefaultEventHandler) > > > 5) send more messages > 6) consumer receives all the messages (but the first one). > > > The same issue appears when you first start the producer and then the > consumer (on a new topic). > -- -- Guozhang
