Re: is task reassignment possible

2022-01-10 Thread DONG, Weike
Hi Qin, Thanks for bringing up this issue. AFAIK, there is no such mechanism in Flink for dynamic task re-assignment at runtime, as states need to be correctly re-distributed across the nodes, which is highly error-prone and not well-suited for the current computation model. However, if the data-

is task reassignment possible

2022-01-09 Thread Chen Qin
Hi there, We ran multiple large scale applications YARN clusters, one observation were those jobs often CPU skewed due to topology or data skew on subtasks. And for better or worse, the skew leads to a few task managers consuming large vcores while majority task managers consume much less. Our goa