Hi, With Flink 1.8.0 I'm not sure how reliable the backpressure status is in the WebUI when it comes to the Async operators. If I remember correctly until around Flink 1.10 (+/- 2 version) backpressure monitoring was checking for thread dumps stuck in requesting Flink's network memory buffers. If in your job AsyncFunction is the source of a backpressure, it would be skipped and not reported. For analysing backpressure I would highly recommend upgrading to Flink 1.13.x as it has greatly improved tooling for that [1]. And in that version AsynFunctions are definitely handled correctly. Since Flink 1.10 I believe you can use the `isBackPressured` metric. In previous versions you would have to rely on buffer usage metrics as described here [2].
[1] https://flink.apache.org/2021/07/07/backpressure.html [2] https://flink.apache.org/2019/07/23/flink-network-stack-2.html#network-metrics Apart of the back pressure, part of the problem might be simply how long does it take for `Async1` function to return the result. Have you checked that? Isn't it taking a couple of seconds? Best, Piotrek wt., 28 wrz 2021 o 15:55 Sanket Agrawal <sanket.agra...@infosys.com> napisał(a): > Hi All, > > > > I am new to Flink. While developing a Flink application We observed that > our message is taking around 10 seconds between the two Async operators. > Below are the details. > > > > - *Flink Flow*: Kinesis Source -> Process -> Async1 -> Async2 -> > Process -> Kinesis Sink > - *Environment*: Amazon KDA. 1 Kinesis Processing Unit (1vCore & 4GB > ram), and 1 parallelism. > - *Flink Version*: 1.8.0 > - *Backpressure*: Flink dashboard shows that backpressure is *OK.* > - *Input rate: *60 messages per second. > > > > Any kind of pointers/help will be very useful. > > > > Thanks, > > Sanket Agrawal > > >
