It makes sense to support operator parallelism. But just FYI, for sink, you can use sink.parallelism[1] to set sink parallelism.
[1] https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/filesystem/#sink-parallelism Best regards, Yuxia ----- 原始邮件 ----- 发件人: "ron9 liu" <ron9....@gmail.com> 收件人: "dev" <dev@flink.apache.org> 发送时间: 星期二, 2023年 4 月 18日 上午 9:37:05 主题: Re: [DISCUSS FLINKSQL PARALLELISM] Hi, Green Thanks for driving this discussion, in batch mode we have the Adaptive Batch Scheduler which automatically derives operator parallelism based on data volume at runtime, so we don't need to care about the parallelism. However, in stream mode, currently, Flink SQL can only set the parallelism of an operator globally, and many users would like to set the parallelism of an operator individually, which seems to be a pain point at the moment, and it would make sense to support set parallelism at operator granularity. Do you have any idea about the solution for this problem? Best, Ron GREEN <1286649...@qq.com.invalid> 于2023年4月14日周五 16:03写道： > Problem： > > > Currently, FlinkSQL can set a unified parallelism in the job,it > cannot set parallelism for each operator. > This can cause resource waste On the occasion of high > parallelism and small data volume.there may also be too many small > file for writing HDFS Scene. > > > Solution： > I can modify FlinkSQL to support operator parallelism.Is it meaningful to > do this？Let's discuss.
