We were using Flink 1.16. I did some research and it looks like it is fixed <https://issues.apache.org/jira/browse/FLINK-30895> in Flink 1.17.
On Thu, Jul 27, 2023 at 9:43 PM Weihua Hu <huweihua....@gmail.com> wrote: > Hi, > > Which Flink version are you using? I haven't reproduced this issue by > using the master branch. > > > Best, > Weihua > > > On Tue, Jul 25, 2023 at 2:56 AM Allen Wang <allenxw...@gmail.com> wrote: > >> Hello, >> >> Our job has operators of source -> sink -> global committer. We have >> created two slot sharing groups, one for source and sink and one for global >> committer. The global committer has specified max parallelism of 1. No max >> parallelism set with the source/sink while there is a system level default >> parallelism set to be the total number of task slots. >> >> After we enabled reactive mode scaling, we found that the parallelism for >> source/sink calculated by the adaptive scheduler is only half of the total >> task slots. For example, if we have 8 slots, the derived parallelism is 4 >> meaning source/sink would use only 4 slots, and the global committer uses 1 >> slot. So we have 3 idle slots which is a waste. >> >> Any ideas how we can avoid those idle slots in this setting? >> >> Thanks, >> Allen >> >> >> >>