Re: Weird behaviour after change sources in a job.

2018-09-13 Thread Oleksandr Nitavskyi
September 2018 at 11:15 To: Oleksandr Nitavskyi Cc: R&D/Product Engineering/PRIME/Delight , "gor...@data-artisans.com" , Juan Gentile , "user@flink.apache.org" Subject: Re: Weird behaviour after change sources in a job. Hi Oleksandr, The mapping of state to operator is d

Re: Weird behaviour after change sources in a job.

2018-09-13 Thread Fabian Hueske
ndr > > > > *From: *Dawid Wysakowicz > *Date: *Thursday, 13 September 2018 at 09:59 > *To: *Juan Gentile , > "user@flink.apache.org" > > *Cc: *R&D/Product Engineering/PRIME/Delight > , > > *Subject: *Re: Weird behaviour after change source

Re: Weird behaviour after change sources in a job.

2018-09-13 Thread Dawid Wysakowicz
gt; *From: *Dawid Wysakowicz > *Date: *Thursday, 13 September 2018 at 09:59 > *To: *Juan Gentile , "user@flink.apache.org" > > *Cc: *R&D/Product Engineering/PRIME/Delight , > > *Subject: *Re: Weird behaviour after change sources in a job. > >   > > Hi

Re: Weird behaviour after change sources in a job.

2018-09-13 Thread Oleksandr Nitavskyi
topics which are different from the currently consumed one. Thank you Oleksandr From: Dawid Wysakowicz Date: Thursday, 13 September 2018 at 09:59 To: Juan Gentile , "user@flink.apache.org" Cc: R&D/Product Engineering/PRIME/Delight , Subject: Re: Weird behaviour after change so

Re: Weird behaviour after change sources in a job.

2018-09-13 Thread Dawid Wysakowicz
Hi Juan, I think this is somehow expected behaviour. Flink, in order to provide proper processing semantics keeps track of partitions offsets internally, and checkpoints those offsets. FlinkKafkaConsumer supports also new partitions discovery. Having in mind both of those features, if you restart