Thank you, Some parts that stick out >The Flink Kafka Consumer allows configuring the behaviour of how offsets are committed back to Kafka brokers. Note that the Flink Kafka Consumer does not rely on the committed offsets for fault tolerance guarantees. The committed offsets are only a means to expose the consumer’s progress for monitoring purposes.
I read this as, "The offsets committed to Kafka are ignored, the offsets committed within a checkpoint are used". >With Flink’s checkpointing enabled, the Flink Kafka Consumer will consume records from a topic and periodically checkpoint all its Kafka offsets, together with the state of other operations. In case of a job failure, Flink will restore the streaming program to the state of the latest checkpoint and re-consume the records from Kafka, starting from the offsets that were stored in the checkpoint. This seems to say something similar. So from the sounds of things, regardless of the consumer group's offsets, it will always start from a checkpoint or savepoints offsets if there are some (unless checkpointing offsets is turned off). Is this interpretation correct? Thanks! On Mon, Jan 18, 2021 at 3:23 AM Piotr Nowojski <pnowoj...@apache.org> wrote: > Hi Rex, > > I believe this section answers your question [1] > > Piotrek > > [1] > https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/kafka.html#kafka-consumers-start-position-configuration > > pon., 18 sty 2021 o 09:00 赵一旦 <hinobl...@gmail.com> napisał(a): > >> If you changed the consumer group in your new job, the group id will be >> the new one you set. >> The job will continue to consumer the topics from the >> savepoint/checkpoint you specified no matter whether the group id is the >> original one? >> >> Rex Fenley <r...@remind101.com> 于2021年1月18日周一 下午12:53写道： >> >>> Hello, >>> >>> When using the Kafka consumer connector, if we restore a from a >>> checkpoint or savepoint using a differently named consumer group than the >>> one we originally ran a job with will it still pick up exactly where it >>> left off or are you locked into using the same consumer group as before? >>> >>> Thanks! >>> >>> -- >>> >>> Rex Fenley | Software Engineer - Mobile and Backend >>> >>> >>> Remind.com <https://www.remind.com/> | BLOG <http://blog.remind.com/> >>> | FOLLOW US <https://twitter.com/remindhq> | LIKE US >>> <https://www.facebook.com/remindhq> >>> >> -- Rex Fenley | Software Engineer - Mobile and Backend Remind.com <https://www.remind.com/> | BLOG <http://blog.remind.com/> | FOLLOW US <https://twitter.com/remindhq> | LIKE US <https://www.facebook.com/remindhq>
