Hi Akshay,
For your questions,
1. One main purpose of maxParallelism is to decide the count of
keyGroup. keyGroup is the bucket for keys when doing keyBy partitioning.
So a larger maxParallelism indicates a finer granularity for key
distribution. No matter it's a stateful operator or not.
2. You
Hi
So we are running a beam pipeline that uses flink as its execution engine. We
are currently on flink1.8
So per the flink documentation I see that there is an option that allows u to
set
Parallelism
and
maxParallelism.
We actually want to set both so that we can dynamically scale the pipeline
