Re: Strange Kafka consumer behaviour

2018-03-19 Thread Tzu-Li (Gordon) Tai
Hi Gyula, Are you using Flink 1.4.x, and have partition discovery enabled? If yes, then both the state of previously existing topics, as well as partitions of the newly specified topics will be consumed. Cheers, Gordon On Tue, Mar 20, 2018 at 6:01 AM, Ankit Chaudhary wrote: > Did you changed t

Re: Strange Kafka consumer behaviour

2018-03-19 Thread Ankit Chaudhary
Did you changed the kafka group id? On Mon, Mar 19, 2018 at 9:54 PM, Aljoscha Krettek wrote: > I think it's because it's just taking the state for the existing topics > and starts reading from that. > > A workaround could be to change the uid of the operator to avoid reading > the old state. > >

Re: Strange Kafka consumer behaviour

2018-03-19 Thread Aljoscha Krettek
I think it's because it's just taking the state for the existing topics and starts reading from that. A workaround could be to change the uid of the operator to avoid reading the old state. Aljoscha > On 21. Feb 2018, at 02:21, Gyula Fóra wrote: > > Hi, > I have observed a weird behaviour wh

Strange Kafka consumer behaviour

2018-02-21 Thread Gyula Fóra
Hi, I have observed a weird behaviour when changing kafka topics when restoring from a checkpoint. It seems that the job started consuming both the topics from the state, and the new topic that I assigned. This happened while changing from kafka 08 to kafka 10. Is this expected? Thanks, Gyula