Hi, We are exploring options to support canary upgrades for our Flink pipelines and have considered the following approach.
1. *Stateless Pipelines*: For stateless pipelines, we define two clusters, job-v1 and job-v2, using the same consumer group to avoid event duplication. To control the data handled by each job, we set different parallelism levels on the consumer. This approach works well as long as no state is involved. 2. *Stateful Pipelines*: In the case of stateful pipelines, we need to ensure that each cluster receives events that are properly partitioned by key. The solution we've found so far is to explicitly assign partitions per cluster in the consumer definition for each job. Is there an alternative method to manage this, ideally through configuration rather than explicit code changes? We would appreciate any insights or suggestions BTW, we are using flink 1.18 and flink operator 1.7. Thanks Sigalit
