Re: Flink Kafka source getting marked as Idle

2023-06-17 Thread Anirban Dutta Gupta
th the currently supported watermark strategies, the only way for a source subtask to resume activeness is when it reports a new watermark (e.g. when new data is produced). Best, Gordon On Fri, Jun 16, 2023 at 7:10 AM Anirban Dutta Gupta wrote: Hello All, Sorry to be replying to an e

Flink Kafka source getting marked as Idle

2023-06-16 Thread Anirban Dutta Gupta
Hello All, Sorry to be replying to an existing thread for my question. Actually we are also facing the issue of the Flink Kafka source stopping consuming messages completely. It only started consuming messages after we re-submitted the Job. But this happened only once and now it is not getting

Re: Flink source error handling

2023-06-10 Thread Anirban Dutta Gupta
t 11:05 AM Anirban Dutta Gupta wrote: Hello, We are using "FlinkKafkaConsumer011" as a Kafka source consumer for Flink. Please guide on how to implement error handling mechanism for the following: 1. If the subscription to the Kafka topic gets lost, Kafka

Flink source error handling

2023-06-09 Thread Anirban Dutta Gupta
Hello, We are using "FlinkKafkaConsumer011" as a Kafka source consumer for Flink. Please guide on how to implement error handling mechanism for the following: 1. If the subscription to the Kafka topic gets lost, Kafka connection gets disconnected. In this case, is there any mechanism of re-sub

Flink program getting unresponsive suddenly

2023-06-04 Thread Anirban Dutta Gupta
Hello, We have a Flink application with the following architecture: Kafka producers >> Send data to Kakfa topic >> Flink program subscribed to Kafka topic >> Data process and send processed data to ElastcSearch. Few days ago we encountered strange problem on the Flink servers deployed on Pro