Failed Checkpoints when Flink HPA enabled on Kubernetes

2023-01-24 Thread Varun Narayanan Chakravarthy via user
Hello Flink Users, We have enabled Kubernetes HPA for our Flink applications (Task Manager only). Our Flink Applications are running in Reactive Mode. When the Kubernetes controller triggers a scale-up/scale-down of our job's Task Managers, we get an alert for failed checkpoints. Interestingly enou

Default Flink S3 FileSource Questions

2023-09-25 Thread Varun Narayanan Chakravarthy via user
Hello Flink Community, Flink Version: 1.16.1, Zookeeper for HA. My Flink Applications reads raw parquet files hosted in S3, applies transformations and re-writes them to S3, under a different location. Below is my code to read from parquets from S3: ``` final Configuration configuration = new C