Hi Vishal, Flink does checkpoint to Kafka(Offset commits) by default which could be disabled. Look here <https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/kafka.html#kafka-consumers-offset-committing-behaviour-configuration> for more information.
Regards, Nagarjun *Success is not final, failure is not fatal: it is the courage to continue that counts. * *- Winston Churchill - * On Mon, Feb 4, 2019 at 3:31 PM Vishal Santoshi <vishal.santo...@gmail.com> wrote: > A simple query > > Does Flink' KafkaConnector flush the current offsets to kafka on a SP ? > Note that the I do koow that Flink consumes data from Kafka topics and > periodically checkpoints using Flink's distributed checkpointing mechanism. > In case of failure, Flink will restore the records from checkpoint > directory and will start reading data from Kafka offset after that. In this > case however I want to restart a radically different pipe but from an > offset that is part of the SP and the hope is that I could just depend on > the kafka offsets of the consumer group. > > Thanks >
