Hi,
Flink doesn't do any special failure-handling or retry logic, so it’s up
to how the KafkaConsumer is configured via properties. In general Flink
doesn’t try to be smart: when something fails an exception fill bubble
up that will fail this execution of the job. If checkpoints are enabled
this will trigger a restore, this is controlled by the restart strategy.
If that eventually gives up the job fill go to “FAILED” and stop.
This is the relevant section of the docs:
https://ci.apache.org/projects/flink/flink-docs-stable/dev/task_failure_recovery.html
Best,
Aljoscha
On 15.07.20 17:42, Nick Bendtner wrote:
Hi guys,
I want to know what is the default behavior of Kafka source when a kafka
cluster goes down during streaming. Will the job status go to failing or is
the exception caught and there is a back off before the source tries to
poll for more events ?
Best,
Nick.
