RE: Kafka offset commits

2017-04-21 Thread Gwenhael Pasquiers
restarting. From: Tzu-Li (Gordon) Tai [mailto:tzuli...@apache.org] Sent: mercredi 19 avril 2017 11:39 To: user@flink.apache.org Subject: Re: Kafka offset commits Thanks for the clarification Aljoscha! Yes, you cannot restore from a 1.0 savepoint in Flink 1.2 (sorry, I missed the “1.0” part on my

Re: Kafka offset commits

2017-04-19 Thread Tzu-Li (Gordon) Tai
don) Tai [mailto:tzuli...@apache.org]  Sent: lundi 17 avril 2017 07:40 To: user@flink.apache.org Subject: Re: Kafka offset commits   Hi,   The FlinkKafkaConsumer in 1.2 is able to restore from older version state snapshots and bridge the migration, so there should be no problem in reading the offs

Re: Kafka offset commits

2017-04-19 Thread Aljoscha Krettek
s, but it looks like it did not restore correctly the offset and it > started consuming from the oldest offset, creating duplicated data (the kafka > queue has over a week of buffer). > B.R. > > From: Tzu-Li (Gordon) Tai [mailto:tzuli...@apache.org] > Sent: lundi 17 avril 20

RE: Kafka offset commits

2017-04-18 Thread Gwenhael Pasquiers
the oldest offset, creating duplicated data (the kafka queue has over a week of buffer). B.R. From: Tzu-Li (Gordon) Tai [mailto:tzuli...@apache.org] Sent: lundi 17 avril 2017 07:40 To: user@flink.apache.org Subject: Re: Kafka offset commits Hi, The FlinkKafkaConsumer in 1.2 is able to restore

Re: Kafka offset commits

2017-04-16 Thread Tzu-Li (Gordon) Tai
Hi, The FlinkKafkaConsumer in 1.2 is able to restore from older version state snapshots and bridge the migration, so there should be no problem in reading the offsets from older state. The smallest or highest offsets will only be used if the offset no longer exists due to Kafka data retention s