Re: Flink stress testing and metrics

It is working with FLINK-7608 , but just to know : how to implement it with slf4jReporter ? I didn't find an example ! 2017-11-27 17:33 GMT+01:00 Chesnay Schepler : > The most reliable way to see the latency metric is configure a metric > reporter >

Re: Flink stress testing and metrics

The most reliable way to see the latency metric is configure a metric reporter . However, only some reporters can properly work with the latency metric (about to change with FLINK-7608 though!). The

Re: Flink stress testing and metrics

Thanks Aljoscha, as I see it is not fixed yet ( In Progress ) can you give me another solution to visualize the latency or exporting them to a file , ... I want to get the latency in any way: file, graph, ... just to get idea of the latency. Regards. 2017-11-27 13:17 GMT+01:00 Aljoscha Krettek :

Re: Flink stress testing and metrics

Hi, This is a known issue: the latency metrics are reported in a format that the web dashboard does not understand. This is the Jira issue for fixing it: https://issues.apache.org/jira/browse/FLINK-7608 Best, Aljoscha > On 27. Nov 2017, at 0

Re: Flink stress testing and metrics

Thanks Timo for your answer. Can any one else confirm the bug ? 2017-11-23 9:26 GMT+01:00 Timo Walther : > Yes, I agree that this looks like a bug. You can open an issue about that. > Maybe with a small reproduceble example to give others the chance to fix it. > > > Am 11/22/17 um 10:18 PM schrie

Re: Flink stress testing and metrics

Yes, I agree that this looks like a bug. You can open an issue about that. Maybe with a small reproduceble example to give others the chance to fix it. Am 11/22/17 um 10:18 PM schrieb Ladhari Sadok: Normally it should return 0ms in case of no latency not NaN, and my real data size is 1kb, but

Re: Flink stress testing and metrics

Normally it should return 0ms in case of no latency not NaN, and my real data size is 1kb, but for now I'm using 200 bytes, I will try it with the real size later. For the data generator, it is an infinite for loop. Thanks. 2017-11-22 18:11 GMT+01:00 Timo Walther : > At a first glance I would s

Re: Flink stress testing and metrics

At a first glance I would say that your data size is very small. Flink is able to process millions of records on a single machine. It might be that the records are produced to quickly to be used for latency measuring. Is you data generator never-ending? Am 11/22/17 um 4:13 PM schrieb Ladhari

Re: Flink stress testing and metrics

Thanks Timo for your answer. I have tried to setLatencyTrackingInterval(1000) but I have got the same result ( latency : NaN ) My Flink Job is a geofencing pattern : - [Latitude,Langitude ] < IN | OUT > Location ? Send Notification : None In my stress test I'm using data that always send no

Re: Flink stress testing and metrics

Hi Sadok, it would be helpful if you could tell us a bit more about your job. E.g. a skewed key distribution where keys are only sent to one third of your operators can not use your CPUs full capabilities. The latency tracking interval is in milliseconds. Can you try if 1000 would fix your p

Flink stress testing and metrics

Hi All, I want to do a stress testing of my Flink app implementation: event generation with ParallelSourceFunction then measuring the latency ,throughput, CPU & memry leak ... But when testing, I noticed that : - the maximum of CPU usage is 30-33% - latency is always NaNd NaNh in the dashb