Hi team, I have a Flink cluster setup with 1 JobManager and 2 TaskManagers (TM1 with 6 slots and TM2 with 5 slots, total 11 slots). My Kafka cluster has 2 brokers.
I’m running a job where the source is Kafka and the sink sends POST requests to an HTTP endpoint. While testing different job parallelism levels, I noticed the following behavior in terms of slot allocation: P=5 → TM1: 3, TM2: 2 → Even load across both TMs P=6 → TM1: 3, TM2: 3 → Load goes only to one TM P=7 → TM1: 4, TM2: 3 → Even load across both TMs P=8 → TM1: 4, TM2: 4 → Load goes only to one TM P=9 → TM1: 5, TM2: 4 → Load goes only to one TM P=10 → TM1: 5, TM2: 5 → Even load across both TMs P=11 → TM1: 6, TM2: 5 → Even load across both TMs So, for parallelism values 5, 7, 10, and 11, the tasks are distributed evenly across both TaskManagers. But for 6, 8, and 9, the tasks are placed only on one TaskManager. Why does Flink behave this way, and what could be the reason for this uneven task distribution? Thanks in Advance, Arjun S
