The main benefit of letting Flink keep the offsets is that you get exactly
once semantics (with the offsets in Flink state, it is aligned with all
your other state).
When storing the offsets in Kafka, you get at least once semantics (= you
are seeing some messages twice on restore / when continuing
What are the pros and cons of Kafka offset keeping vs Flink offset
keeping? Is one more reliable than the other? Personally I prefer having
flink manage it due to it being intrinsically tied to its checkpointing
mechanism. But interested to learn from others experiences.
Thanks
Tim
On Thu, F
Thanks Theo !
From: "theo.diefent...@scoop-software.de"
Date: Thursday, 13 February 2020 at 12:13 AM
To: "Hegde, Mahendra" , "user@flink.apache.org"
Subject: [External] AW: How Flink Kafka Consumer works when it restarts
Hi Mahendra,
Flink will regularly create checkpoints or manually trigger
