Hello, Our job has operators of source -> sink -> global committer. We have created two slot sharing groups, one for source and sink and one for global committer. The global committer has specified max parallelism of 1. No max parallelism set with the source/sink while there is a system level default parallelism set to be the total number of task slots.
After we enabled reactive mode scaling, we found that the parallelism for source/sink calculated by the adaptive scheduler is only half of the total task slots. For example, if we have 8 slots, the derived parallelism is 4 meaning source/sink would use only 4 slots, and the global committer uses 1 slot. So we have 3 idle slots which is a waste. Any ideas how we can avoid those idle slots in this setting? Thanks, Allen
