Hi Qin,
Thanks for bringing up this issue. AFAIK, there is no such mechanism in
Flink for dynamic task re-assignment at runtime, as states need to be
correctly re-distributed across the nodes, which is highly error-prone and
not well-suited for the current computation model.
However, if the data-
Hi there,
We ran multiple large scale applications YARN clusters, one observation
were those jobs often CPU skewed due to topology or data skew on subtasks.
And for better or worse, the skew leads to a few task managers consuming
large vcores while majority task managers consume much less. Our goa