Hi all,
Lately I've added more resources to my Flink cluster which required a restart 
of all apps.
>From the cluster side, the only change I made, is to add more task slots.
On the cluster I have a streaming app that consumes from Kafka and sinks to 
files.
I noticed that since the restart, the applications "warmup" has impacted 
dramatically.
Before, the change it took few minutes for the app to start and consume 
normally (from my point of view, normally is a stable rate) - from 0 to 16K 
events per second in 4 minutes.
Now, after the change, it takes hours till it stabilizes on the normal 
processing rate- from 0 to 12K events per second in 3 hours.
The data source behavior hasn't changed (same incoming rate, partitions, 
servers etc.).
I am aware to the backpressure mechanism in Flink, but it seems like it works 
too slow here.
Is there a way to speed or control it?

Thanks a lot
Eyal Peer

Reply via email to