Hi Dhruv,
> On May 12, 2016, at 11:07pm, Dhruv Gohil <[email protected]> wrote:
>
> Hi Prateek,
>
>
> <https://github.com/dataArtisans/yahoo-streaming-benchmark/blob/master/flink-benchmarks/src/main/java/flink/benchmark/utils/ThroughputLogger.java>https://github.com/dataArtisans/yahoo-streaming-benchmark/blob/master/flink-benchmarks/src/main/java/flink/benchmark/utils/ThroughputLogger.java
>
> <https://github.com/dataArtisans/yahoo-streaming-benchmark/blob/master/flink-benchmarks/src/main/java/flink/benchmark/utils/ThroughputLogger.java>
> https://github.com/dataArtisans/yahoo-streaming-benchmark/blob/master/flink-benchmarks/src/main/java/flink/benchmark/utils/AnalyzeTool.java
>
> <https://github.com/dataArtisans/yahoo-streaming-benchmark/blob/master/flink-benchmarks/src/main/java/flink/benchmark/utils/AnalyzeTool.java>
>
> help you measure throughput and latency both, from within topology.
The AnalyzeTool is processing a file that has lines which match the pattern:
Pattern latencyPattern = Pattern.compile(".*Latency ([0-9]+) ms.*”);
This isn’t something created by the ThroughputLogger.
Who generates these files, and how do they calculate the latency?
Thanks,
— Ken
> On Thursday 12 May 2016 11:21 PM, Konstantin Knauf wrote:
>> Hi Prateek,
>>
>> regarding throughput, what about simply filling the input Kafka topic
>> with some (a lot) of messages and monitor (e.g.
>> http://quantifind.github.io/KafkaOffsetMonitor/
>> <http://quantifind.github.io/KafkaOffsetMonitor/>) how quickly Flink can
>> work the lag off. The messages should be representative of your use
>> case, of course.
>>
>> Latency is harder, I think, and I would also be interested in the
>> approaches of others to measure latency in Flink.
>>
>> To some extend, you could do it by adding some logging inside Flink, but
>> this effects latency and only measure latency whithin Flink (excluding
>> reading from source and writing to sink).
>>
>> Cheers,
>>
>> Konstantin
>>
>> On 12.05.2016 18:57, prateekarora wrote:
>>> Hi
>>>
>>> How can i measure throughput and latency of my application in flink 1.0.2
>>> ?
>>>
>>> Regards
>>> Prateek
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/How-to-measure-Flink-performance-tp6741p6863.html
>>>
>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/How-to-measure-Flink-performance-tp6741p6863.html>
>>> Sent from the Apache Flink User Mailing List archive. mailing list archive
>>> at Nabble.com.
>>>
>
--------------------------
Ken Krugler
+1 530-210-6378
http://www.scaleunlimited.com
custom big data solutions & training
Hadoop, Cascading, Cassandra & Solr
