RE: Event is taking a lot of time between the operators

2021-09-29 Thread Sanket Agrawal
Thank you @Piotr Nowojski<mailto:pnowoj...@apache.org> for helping me. From: Piotr Nowojski Sent: Wednesday, September 29, 2021 12:53 PM To: Sanket Agrawal Cc: Ragini Manjaiah ; user@flink.apache.org Subject: Re: Event is taking a lot of time between the operators [**EXTERNAL EMAIL

Re: Event is taking a lot of time between the operators

2021-09-29 Thread Piotr Nowojski
ommendations on improving the approach. > > > > Thanks, > > Sanket Agrawal > > > > *From:* Ragini Manjaiah > *Sent:* Wednesday, September 29, 2021 11:17 AM > *To:* Sanket Agrawal > *Cc:* Piotr Nowojski ; user@flink.apache.org > *Subject:* Re: Event is taki

RE: Event is taking a lot of time between the operators

2021-09-28 Thread Sanket Agrawal
ra...@infosys.com>> Cc: user@flink.apache.org<mailto:user@flink.apache.org> Subject: Re: Event is taking a lot of time between the operators [**EXTERNAL EMAIL**] Hi, With Flink 1.8.0 I'm not sure how reliable the backpressure status is in the WebUI when it comes to the Async ope

Re: Event is taking a lot of time between the operators

2021-09-28 Thread Ragini Manjaiah
tly) is available for the next message as soon as it > calls CompletableFuture.supplyAsync on the current message. > > > > Thanks, > > Sanket Agrawal > > > > *From:* Piotr Nowojski > *Sent:* Tuesday, September 28, 2021 8:02 PM > *To:* Sanket Agrawal

RE: Event is taking a lot of time between the operators

2021-09-28 Thread Sanket Agrawal
Event is taking a lot of time between the operators [**EXTERNAL EMAIL**] Hi, With Flink 1.8.0 I'm not sure how reliable the backpressure status is in the WebUI when it comes to the Async operators. If I remember correctly until around Flink 1.10 (+/- 2 version) backpressure monitoring w

Re: Event is taking a lot of time between the operators

2021-09-28 Thread Piotr Nowojski
Hi, With Flink 1.8.0 I'm not sure how reliable the backpressure status is in the WebUI when it comes to the Async operators. If I remember correctly until around Flink 1.10 (+/- 2 version) backpressure monitoring was checking for thread dumps stuck in requesting Flink's network memory buffers. If

Event is taking a lot of time between the operators

2021-09-28 Thread Sanket Agrawal
Hi All, I am new to Flink. While developing a Flink application We observed that our message is taking around 10 seconds between the two Async operators. Below are the details. * Flink Flow: Kinesis Source -> Process -> Async1 -> Async2 -> Process -> Kinesis Sink * Environment: Amazon