Monika Hristova created FLINK-27481:
---------------------------------------
Summary: Flink checkpoints are very slow after upgrading from
Flink 1.13.1 to Flink 1.14.3
Key: FLINK-27481
URL: https://issues.apache.org/jira/browse/FLINK-27481
Project: Flink
Issue Type: Bug
Components: Runtime / Checkpointing
Affects Versions: 1.14.4
Reporter: Monika Hristova
I am experiencing extremely slow checkpoints after upgrading flink version from
1.13.1 to 1.14.3. The checkpointing is slow because of the sink/source. Example
[!https://i.stack.imgur.com/oChk9.png!|https://i.stack.imgur.com/oChk9.png]
This source is a kafka topic that have some messages inside and still, it might
take up to minutes to be checkpointed.
I am using incremental checkpoints, unaligned checkpoints are not enabled.
Configurations are the default ones, and this source was checkpointed for
milliseconds with flink 1.13.1 and now it times out(the timeout is 3 minutes)
for every second chechpoint.
[!https://i.stack.imgur.com/kTbSx.png!|https://i.stack.imgur.com/kTbSx.png]
[FlinkKafkaConsumer|https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer.html]/[Producer|https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaProducer.html]
are used which are deprecated.
After removing the topic with the few configurational messages the checkpoint
time improved drastically but it still takes around a minute for the checkpoint
due to alignment duration in the kafka sinks.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
