Latency Monitoring in Flink application

Hi All, I'm looking for help regarding latency monitoring. Let's say I have a simple streaming data flow with the following operators: FlinkKafkaConsumer -> Map -> print. In case I want to measure a latency of records processing in my dataflow, what would be the best opportunity? I want to get t

Asynchronous I/O

Hi All, I have a flow which contains few external REST calls, in few different operators. As I read in documentation, using AsyncFunction is the recommended way to execute these calls. Few questions regarding that: 1. Can someone explain better the 'capacity' parameter? How it affects latency/

Flink Application JAR

Hi All, We have an application which contains few modules (one for the main application with the "driver" and few other for the operators implementation, configuration etc.). At the moment we are building fat jar contains all the modules and all the 3rd parties dependencies. As we continue deve

RE: Flink error handling

Hi, Do you have any progress with that? -Original Message- From: Steven Nelson Sent: Tuesday, June 18, 2019 7:02 PM To: user@flink.apache.org Subject: Flink error handling Hello! We are internally having a debate on how best to handle exceptions within our operators. Some advocate fo

RE: Flink Application JAR

t the whole project code. And we want to work in a more modular way. As you said, I also think the size of the jar is less important than what I wrote above. Thanks. From: Timothy Victor Sent: Wednesday, July 3, 2019 2:31 PM To: Halfon, Roey Cc: user ; Cohen, Inbal ; Dvir, Liran ; Hasenfeld

Tracking message processing in my application

Hi, We are looking for a monitoring solution for our dataflow - Track the progress of incoming messages while they are processed. I'll clarify - we want to build some service which will show status for each incoming message. And in case of failures to give some detailed information. I thought ab