In fact "*Checkpointing enabled:* if checkpointing is enabled, the Flink Kafka Consumer will commit the offsets stored in the checkpointed states when the checkpoints are completed. This ensures that the committed offsets in Kafka brokers is consistent with the offsets in the checkpointed states." confirms that.
On Mon, Feb 4, 2019 at 7:29 PM Vishal Santoshi <vishal.santo...@gmail.com> wrote: > So it does that also commit the offsets to *kafka* on* > checkpoint/savepoint *as well to it's own distributed state ? Just wanted > to confirm. > > On Mon, Feb 4, 2019 at 6:56 PM Nagarjun Guraja <nagar...@gmail.com> wrote: > >> Hi Vishal, >> >> Flink does checkpoint to Kafka(Offset commits) by default which could be >> disabled. Look here >> <https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/kafka.html#kafka-consumers-offset-committing-behaviour-configuration> >> for more information. >> >> Regards, >> Nagarjun >> >> *Success is not final, failure is not fatal: it is the courage to >> continue that counts. * >> *- Winston Churchill - * >> >> >> On Mon, Feb 4, 2019 at 3:31 PM Vishal Santoshi <vishal.santo...@gmail.com> >> wrote: >> >>> A simple query >>> >>> Does Flink' KafkaConnector flush the current offsets to kafka on a SP ? >>> Note that the I do koow that Flink consumes data from Kafka topics and >>> periodically checkpoints using Flink's distributed checkpointing mechanism. >>> In case of failure, Flink will restore the records from checkpoint >>> directory and will start reading data from Kafka offset after that. In this >>> case however I want to restart a radically different pipe but from an >>> offset that is part of the SP and the hope is that I could just depend on >>> the kafka offsets of the consumer group. >>> >>> Thanks >>> >>
