[
https://issues.apache.org/jira/browse/KAFKA-3811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15326855#comment-15326855
]
aarti gupta edited comment on KAFKA-3811 at 6/13/16 5:51 AM:
-------------------------------------------------------------
Yes indeed, as per
http://www.brendangregg.com/blog/2014-06-09/java-cpu-sampling-using-hprof.html
Tried to do something similar using simpleBenchmark (on a different fix) and
yourkit profiler as [~gfodor] mentions below, here
https://github.com/apache/kafka/pull/1446#issuecomment-225488213, but not
convinced that the results mean anything conclusive.
thoughts/suggestions on a repeatable/consistent toolset ?
Also, Like the idea of 3...Add a general purpose feature to the metrics library
and use it across the producer, consumer, and streams.
But before we refactor the existing library want a reproducible test, any
suggestions on a sceanrio other than SimpleBenchmark?
was (Author: aartigupta):
Yes indeed, as per
http://www.brendangregg.com/blog/2014-06-09/java-cpu-sampling-using-hprof.html
Tried to do something similar using simpleBenchmark (on a different fix) and
yourkit profiler as [~gfodor] mentions below, here
https://github.com/apache/kafka/pull/1446#issuecomment-225488213, but not
convinced that the results mean anything conclusive.
thoughts/suggestions on a repeatable/consistent toolset
Like the idea of 3>>Add a general purpose feature to the metrics library and
use it across the producer, consumer, and streams.
But before we refactor the existing library want a reproducible test, any
suggestions on a sceanrio other than SimpleBenchmark?
> Introduce Kafka Streams metrics recording levels
> ------------------------------------------------
>
> Key: KAFKA-3811
> URL: https://issues.apache.org/jira/browse/KAFKA-3811
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Greg Fodor
> Assignee: aarti gupta
> Attachments: Muon-Snapshot.zip, Muon-latency.zip, screenshot-1.png,
> screenshot-latency.png
>
>
> Follow-up from the discussions here:
> https://github.com/apache/kafka/pull/1447
> https://issues.apache.org/jira/browse/KAFKA-3769
> The proposal is to introduce configuration to control the granularity/volumes
> of metrics emitted by Kafka Streams jobs, since the per-record level metrics
> introduce non-trivial overhead and are possibly less useful once a job has
> been optimized.
> Proposal from guozhangwang:
> level0 (stream thread global): per-record process / punctuate latency, commit
> latency, poll latency, etc
> level1 (per processor node, and per state store): IO latency, per-record ..
> latency, forward throughput, etc.
> And by default we only turn on level0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
