Hi, Eric Thanks for bringing this discussion. I think it's reasonable to add ''setMaxParallelism" for DataStreamSink.
+1 Best, Weihua On Sat, Apr 22, 2023 at 3:20 AM eric xiao <xiao.eric...@gmail.com> wrote: > Hi there devs, > > I would like to start a discussion thread for FLINK-31873[1]. > > We are in the processing of enabling Flink reactive mode as the default > scheduling mode. While reading configuration docs [2] (I believe it was > also mentioned during one of the training sessions during Flink Forward > 2022), one can/should replace all setParallelism calls with > setMaxParallelism when migrating to reactive mode. > > This currently isn't possible on a sink in a Flink pipeline as we do not > expose a setMaxParallelism on the DataStreamSink class [3]. The underlying > Transformation class does have both a setMaxParallelism and setParallelism > function defined [4], but only setParallelism is offered in the > DataStreamSink class. > > I believe adding setMaxParallelism would be beneficial for not just flink > reactive mode, both modes of running of a flink pipeline (non reactive > mode, flink auto scaling). > > Best, > > Eric Xiao > > [1] https://issues.apache.org/jira/browse/FLINK-31873 > [2] > > https://nightlies.apache.org/flink/flink-docs-release-1.17/docs/deployment/elastic_scaling/#configuration > [3] > > https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/datastream/DataStreamSink.java > [4] > > https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/api/dag/Transformation.java#L248-L285 >
