Re: How Flink Kafka Consumer works when it restarts

2020-02-15 Thread Theo Diefenthal
;Mahendra Hegde" , "Theo Diefenthal" , "user" Gesendet: Freitag, 14. Februar 2020 15:26:44 Betreff: Re: AW: How Flink Kafka Consumer works when it restarts The main benefit of letting Flink keep the offsets is that you get exactly once semantics (with the offsets in Fli

Re: AW: How Flink Kafka Consumer works when it restarts

2020-02-14 Thread Robert Metzger
iefent...@scoop-software.de> >> *Date: *Thursday, 13 February 2020 at 12:13 AM >> *To: *"Hegde, Mahendra" , " >> user@flink.apache.org" >> *Subject: *[External] AW: How Flink Kafka Consumer works when it restarts >> >> >> >> Hi Ma

Re: AW: How Flink Kafka Consumer works when it restarts

2020-02-13 Thread Timothy Victor
unicate with Kafka and ask kafka to poll messages > starting from those offsets. > > > > Best regards > > Theo > > > > > Von meinem Huawei-Telefon gesendet > > > > Ursprüngliche Nachricht > Von: "Hegde, Mahendra" > Datum: Mi.

Re: AW: How Flink Kafka Consumer works when it restarts

2020-02-12 Thread Hegde, Mahendra
Thanks Theo ! From: "theo.diefent...@scoop-software.de" Date: Thursday, 13 February 2020 at 12:13 AM To: "Hegde, Mahendra" , "user@flink.apache.org" Subject: [External] AW: How Flink Kafka Consumer works when it restarts Hi Mahendra, Flink will regularly

AW: How Flink Kafka Consumer works when it restarts

2020-02-12 Thread theo.diefent...@scoop-software.de
17:50 An: user@flink.apache.org Betreff: How Flink Kafka Consumer works when it restarts Hi All, I am bit confused on Flink kafka consumer working. I read that Flink stores the kafka message offset in checkpoint and uses it in case if it restarts. Question is when exactly Flink is committing a

How Flink Kafka Consumer works when it restarts

2020-02-12 Thread Hegde, Mahendra
Hi All, I am bit confused on Flink kafka consumer working. I read that Flink stores the kafka message offset in checkpoint and uses it in case if it restarts. Question is when exactly Flink is committing about successful consumption confirmation to kafka broker? And when Flink job restarts will