+ user@f.a.o (adding the conversation back to the user mailing list)
On Fri, Mar 12, 2021 at 6:06 AM Kevin Kwon wrote:
> Thanks Tzu-Li
>
> Interesting algorithm. Is consumer offset also committed to Kafka at the
> last COMMIT stage after the checkpoint has completed?
>
Flink does commit the of
Hi Kevin,
Perhaps the easiest way to answer your question, is to go through how the
exactly-once FlinkKafkaProducer using a 2PC implementation on top of
Flink's checkpointing mechanism.
The phases can be broken down as follows (simplified assuming max 1
concurrent checkpoint and that checkpoint c
Hi team, I just have a bit of confusion where Two Phase Commit and Kafka's
transaction aware producer using transaction.id and enable.autocommit plays
together
what I understand of Flink checkpoint (correct me if I'm wrong) is that it
saves the transaction ID as well as the consumer's commit offse
