Re: Understanding Max Parallelism

2021-03-19 Thread Dawid Wysakowicz
Hi Aeden, The maxParallelism option defines the number of key groups that will be created within the keyed state and thus define the maximum parallelism that a Flink keyed job can scale up to as each key group must be atomically assigned to a single task. You can read more on how the rescaling wor

Understanding Max Parallelism

2021-03-18 Thread Aeden Jameson
I'm trying to get my head around the impact of setting max parallelism. * Does max parallelism primarily serve as a reservation for future increases to parallelism? The reservation being the ability to restore from checkpoints and savepoints after increases to parallelism. * Does it serve as a ru