Hi Weihua, From your below info, it is with the expectation in credit-based flow control.
I guess one of the sink parallelism causes the backpressure, so you will see that there are no available credits on Sink side and the outPoolUsage of Map is almost 100%. It really reflects the credit-based states in the case of backpressure. If you want to analyze the root cause of backpressure, you can trace the task stack of respective Sink parallelism to find which operation costs much, then you can increase the parallelism or improve the UDF(if have bottleneck) to have a try. In addition, i am not sure why you choose rescale to shuffle data among operators. The default forward mode can gain really good performance by default if you adjusting the same parallelism among them. Best, Zhijiang ------------------------------------------------------------------ From:Weihua Hu <huweihua....@gmail.com> Send Time:2020年5月24日(星期日) 18:32 To:user <user@flink.apache.org> Subject:Singal task backpressure problem with Credit-based Flow Control Hi, all I ran into a weird single Task BackPressure problem. JobInfo: DAG: Source (1000)-> Map (2000)-> Sink (1000), which is linked via rescale. Flink version: 1.9.0 There is no related info in jobmanager/taskamanger log. Through Metrics, I see that Map (242) 's outPoolUsage is full, but its downstream Sink (121)' s inPoolUsage is 0. After dumping the memory and analyzing it, I found: Sink (121)'s RemoteInputChannel.unannouncedCredit = 0, Map (242)'s CreditBasedSequenceNumberingViewReader.numCreditsAvailable = 0. This is not consistent with my understanding of the Flink network transmission mechanism. Can someone help me? Thanks a lot. Best Weihua Hu