If you set auto.offset.reset to smallest, the consumer should pick up the
first message on new topics.
Thanks,
Jun
On Thu, Mar 13, 2014 at 2:42 AM, David Morales de Frías <
dmora...@paradigmatecnologico.com> wrote:
> Hi there,
>
> A few days ago i have written about an issue (0.8.0) when auto-
I think it is worth fixing
https://issues.apache.org/jira/browse/KAFKA-1006since the behavior for
new topic discovery is broken in the new consumer.
Even if the consumer sets auto.offset.reset=largest, it is very unintuitive
to apply that behavior to the discovery of new topics. I think that new
to
Hi David,
This issue should only happen if the following conditions are satisfied
1) The consumer does not have any offset stored in ZK for this topic when
it first fetches messages for it.
2) The consumer's default offset reset value is "latest".
3) After the topic is created, it will take a bit
One system I am working on we have something similar however we are
creating topics like how the TopicCommand does but different. We keep
those operations persisted in a meta store so consumers that have to-do
something know that everything is all ready for them (as often it is not
just Kafka) for
Hello Guozhang,
We are using Kafka to connect several actors in our project (osgi, storm,
vert.x web app...), all of them being consumers and producers of different
topics. These services can go up and down in different moments.
Anyway, the issue is the same when you first start the consumer and
Hello David,
This is a known issue with consumer starting before the topic is created:
https://issues.apache.org/jira/browse/KAFKA-1006
And we are still figuring the right way to fix it. Could you describe your
use cases where you need to start the consumer beforehand?
Guozhang
On Thu, Mar 13
