We were using Flink 1.16. I did some research and it looks like it is fixed
<https://issues.apache.org/jira/browse/FLINK-30895> in Flink 1.17.


On Thu, Jul 27, 2023 at 9:43 PM Weihua Hu <huweihua....@gmail.com> wrote:

> Hi,
>
> Which Flink version are you using? I haven't reproduced this issue by
> using the master branch.
>
>
> Best,
> Weihua
>
>
> On Tue, Jul 25, 2023 at 2:56 AM Allen Wang <allenxw...@gmail.com> wrote:
>
>> Hello,
>>
>> Our job has operators of source -> sink -> global committer. We have
>> created two slot sharing groups, one for source and sink and one for global
>> committer. The global committer has specified max parallelism of 1. No max
>> parallelism set with the source/sink while there is a system level default
>> parallelism set to be the total number of task slots.
>>
>> After we enabled reactive mode scaling, we found that the parallelism for
>> source/sink calculated by the adaptive scheduler is only half of the total
>> task slots. For example, if we have 8 slots, the derived parallelism is 4
>> meaning source/sink would use only 4 slots, and the global committer uses 1
>> slot. So we have 3 idle slots which is a waste.
>>
>> Any ideas how we can avoid those idle slots in this setting?
>>
>> Thanks,
>> Allen
>>
>>
>>
>>

Reply via email to