Re: Flink Kafka source getting marked as Idle

2023-06-17 Thread Anirban Dutta Gupta
Hello Gordon, Thanks for the explanations. A few follow-up questions: Are you using the DataStream API and using the KafkaSource (and not the older FlinkKafkaConsumer connector)? If that's the case, I believe you always have to set a watermark strategy when adding the KafkaSource to the job t

Re: Flink Kafka source getting marked as Idle

2023-06-17 Thread Tzu-Li (Gordon) Tai
Hi Anirban, > But this happened only once and now it is not getting reproduced at all. This does make it sound a lot like https://issues.apache.org/jira/browse/FLINK-31632. > 1. What is the default watermarking strategy used in Flink. Can I quickly check the default parameters being used by call

Flink Kafka source getting marked as Idle

2023-06-16 Thread Anirban Dutta Gupta
Hello All, Sorry to be replying to an existing thread for my question. Actually we are also facing the issue of the Flink Kafka source stopping consuming messages completely. It only started consuming messages after we re-submitted the Job. But this happened only once and now it is not getting