Re: Restoring from a checkpoint or savepoint on a different Kafka consumer group

Thanks! On Tue, Jan 19, 2021 at 9:47 PM Piotr Nowojski wrote: > Hi Rex, > > Sorry, I might have misled you. I think you were right in your previous > email > > > So from the sounds of things, regardless of the consumer group's > offsets, it will always start from a checkpoint or savepoints offse

Re: Restoring from a checkpoint or savepoint on a different Kafka consumer group

Hi Rex, Sorry, I might have misled you. I think you were right in your previous email > So from the sounds of things, regardless of the consumer group's offsets, it will always start from a checkpoint or savepoints offsets if there are some (unless checkpointing offsets is turned off). > > Is thi

Re: Restoring from a checkpoint or savepoint on a different Kafka consumer group

Thank you, That's unfortunate, because I imagine we often will want to duplicate a job in order to do some testing out-of-bound from the normal job while slightly tweaking / tuning things. Is there any way to transfer offsets between consumer groups? On Tue, Jan 19, 2021 at 5:45 AM Piotr Nowojski

Re: Restoring from a checkpoint or savepoint on a different Kafka consumer group

Hi, > I read this as, "The offsets committed to Kafka are ignored, the offsets committed within a checkpoint are used". yes, exactly > So from the sounds of things, regardless of the consumer group's offsets, it will always start from a checkpoint or savepoints offsets if there are some (unless

Re: Restoring from a checkpoint or savepoint on a different Kafka consumer group

Thank you, Some parts that stick out >The Flink Kafka Consumer allows configuring the behaviour of how offsets are committed back to Kafka brokers. Note that the Flink Kafka Consumer does not rely on the committed offsets for fault tolerance guarantees. The committed offsets are only a means to ex

Re: Restoring from a checkpoint or savepoint on a different Kafka consumer group

Hi Rex, I believe this section answers your question [1] Piotrek [1] https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/kafka.html#kafka-consumers-start-position-configuration pon., 18 sty 2021 o 09:00 赵一旦 napisał(a): > If you changed the consumer group in your new job, the

Re: Restoring from a checkpoint or savepoint on a different Kafka consumer group

If you changed the consumer group in your new job, the group id will be the new one you set. The job will continue to consumer the topics from the savepoint/checkpoint you specified no matter whether the group id is the original one? Rex Fenley 于2021年1月18日周一 下午12:53写道： > Hello, > > When using th

Restoring from a checkpoint or savepoint on a different Kafka consumer group

Hello, When using the Kafka consumer connector, if we restore a from a checkpoint or savepoint using a differently named consumer group than the one we originally ran a job with will it still pick up exactly where it left off or are you locked into using the same consumer group as before? Thanks!