Hi,
Yes, Flink does not rely on consumer offset commits in Kafka / Zookeeper. It
manages offsets as checkpointed state with Flink, and uses those offsets for
exactly-once.
Currently the “auto.offset.reset” is passed into the internally used
KafkaConsumer as a means to define start position, but
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512
I would put this differently: "auto.offset.reset" policy is only used,
if there are no valid committed offsets for a topic.
See here:
http://stackoverflow.com/documentation/apache-kafka/5449/consumer-groups
- -and-offset-management
(don't be confus
You also need to have a new for this to work.
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/How-to-read-from-a-Kafka-topic-from-the-beginning-tp3522p11087.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Hi Will,
In Kafka's consumer configuration [1] there is a configuration parameter
called "auto.offset.reset". Setting it to "smallest" will tell the consumer
to start reading a topic from the smallest available offset.
You can pass the configuration using the properties of the Kafka consumer.
[
Hi,
How can I tell a "FlinkKafkaConsumer" that I want to read from a topic from the
beginning?
Thanks,
Will
