So it does that also commit the offsets to *kafka* on* checkpoint/savepoint *as well to it's own distributed state ? Just wanted to confirm.
On Mon, Feb 4, 2019 at 6:56 PM Nagarjun Guraja <nagar...@gmail.com> wrote: > Hi Vishal, > > Flink does checkpoint to Kafka(Offset commits) by default which could be > disabled. Look here > <https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/kafka.html#kafka-consumers-offset-committing-behaviour-configuration> > for more information. > > Regards, > Nagarjun > > *Success is not final, failure is not fatal: it is the courage to continue > that counts. * > *- Winston Churchill - * > > > On Mon, Feb 4, 2019 at 3:31 PM Vishal Santoshi <vishal.santo...@gmail.com> > wrote: > >> A simple query >> >> Does Flink' KafkaConnector flush the current offsets to kafka on a SP ? >> Note that the I do koow that Flink consumes data from Kafka topics and >> periodically checkpoints using Flink's distributed checkpointing mechanism. >> In case of failure, Flink will restore the records from checkpoint >> directory and will start reading data from Kafka offset after that. In this >> case however I want to restart a radically different pipe but from an >> offset that is part of the SP and the hope is that I could just depend on >> the kafka offsets of the consumer group. >> >> Thanks >> >
