Hi,
I have nothing more to add. You (Dawid) and Vino explained it correctly :)
Piotrek
> On 24 Sep 2018, at 15:16, Dawid Wysakowicz wrote:
>
> Hi Harshvardhan,
>
> Flink won't buffer all the events between checkpoints. Flink uses Kafka's
> transaction, which are committed only on checkpoints
Hi Harshvardhan,
Flink won't buffer all the events between checkpoints. Flink uses
Kafka's transaction, which are committed only on checkpoints, so the
data will be persisted on the Kafka's side, but only available to read
once committed.
I've cced Piotr, who implemented the Kafka 0.11 connector
Hi Harshvardhan,
In fact, Flink does not cache data between two checkpoints. In fact, Flink
only calls different operations at different points in time. These
operations are provided by the Kafka client, so you should have a deeper
understanding of the principles of Kafka producer transactions.
I
Hi,
Can someone please help me understand how does the exactly once semantic
work with Kafka 11 in Flink?
Thanks,
Harsh
On Tue, Sep 11, 2018 at 10:54 AM Harshvardhan Agrawal <
harshvardhan.ag...@gmail.com> wrote:
> Hi,
>
> I was going through the blog post on how TwoPhaseCommitSink function wor
Hi,
I was going through the blog post on how TwoPhaseCommitSink function works
with Kafka 11. One of the things I don’t understand is: What is the
behavior of the Kafka 11 Producer between two checkpoints? Say that the
time interval between two checkpoints is set to 15 minutes. Will Flink
buffer a
