Re: Flink job can't complete initialisation because of millions of savepoint file reads

Hi Alex, Please see my comment here [1]. [1] https://lists.apache.org/thread/h5mv6ld4l2g4hsjszfdos9f365nh7ctf BR, G On Mon, Sep 2, 2024 at 11:02 AM Alex K. wrote: > We have an issue where a savepoint file containing Kafka topic partitions > offsets is requested millions of times from AWS S3.

Flink job can't complete initialisation because of millions of savepoint file reads

We have an issue where a savepoint file containing Kafka topic partitions offsets is requested millions of times from AWS S3. This results in the job crashing and then followed by a restart and crashing again. We have tracked the high number of reads (~3 millions) to Kafka topic partitions (~40k) m