The documentation provides the formula #slots-per-TM^2 * #TMs * 4 to determine the number of network buffers we should configure. The documentation also says, “A logical network connection exists for each point-to-point exchange of data over the network, which typically happens at repartitioning- or broadcasting steps (shuffle phase). In those, each parallel task inside the TaskManager has to be able to talk to all other parallel tasks.” Does that say that each parallel task inside the TaskManager talk to all parallel tasks inside the same TaskManager or to all parallel tasks across all task managers? Intuitively, I would assume the latter, but then wouldn’t the formula for determining the number of network buffers be more along the lines of (#slots-per-TM * #TMs)^2?
Thanks, Ray
