Hi all, Lately I've added more resources to my Flink cluster which required a restart of all apps. >From the cluster side, the only change I made, is to add more task slots. On the cluster I have a streaming app that consumes from Kafka and sinks to files. I noticed that since the restart, the applications "warmup" has impacted dramatically. Before, the change it took few minutes for the app to start and consume normally (from my point of view, normally is a stable rate) - from 0 to 16K events per second in 4 minutes. Now, after the change, it takes hours till it stabilizes on the normal processing rate- from 0 to 12K events per second in 3 hours. The data source behavior hasn't changed (same incoming rate, partitions, servers etc.). I am aware to the backpressure mechanism in Flink, but it seems like it works too slow here. Is there a way to speed or control it?
Thanks a lot Eyal Peer