One thing I would like to see added (if not already present) is the ability to register metrics that are not tied to a component.
As of now, the only non-component metrics are reported by the SystemBolt pseudo-component which feels like a work-around. It reports JVM level metrics like GC time, heap size and other things that aren't associated with a given component. It would be great if application developers could expose similar metrics like this for things like connection pools and other JVM wide objects that aren't unique to a specific component. I don't think this is possible now, is it? On Wed, Apr 20, 2016 at 12:29 AM, Jungtaek Lim <[email protected]> wrote: > Let me start sharing my thought. :) > > 1. Need to enrich docs about metrics / stats. > > In fact, I couldn't see the fact - topology stats are sampled by default > and sample rate is 0.05 - from the docs when I was newbie of Apache > Storm. It made me misleading and made me saying "Why there're difference > between the counts?". I also saw some mails from user@ about same question. > If we include this to guide doc that would be better. > > And Metrics document page > <http://storm.apache.org/releases/1.0.0/Metrics.html> seems not well > written. I think it has appropriate headings but lacks contents on each > heading. > It should be addressed, and introducing some external metrics consumer > plugins (like storm-graphite <https://github.com/verisign/storm-graphite> from > Verisign) would be great, too. > > 2. Need to increase sample rate or (ideally) no sampling at all. > > Let's postpone considering performance hit at this time. > Ideally, we expect precision of metrics gets better when we increase > sample rate. It affects non-gauge kinds of metrics which are counter, and > latency, and so on. > > Btw, I would like to hear about opinions on latency since I'm not an > expert. > Storm provides only average latency and it's indeed based on sample rate. > Do we feel OK with this? If not how much having also percentiles can help > us? > > Thanks, > Jungtaek Lim (HeartSaVioR) > > 2016년 4월 20일 (수) 오전 10:55, Jungtaek Lim <[email protected]>님이 작성: > >> Hi Storm users, >> >> I'm Jungtaek Lim, committer and PMC member of Apache Storm. >> >> If you subscribed dev@ mailing list, you may have seen that recently >> we're addressing the metrics feature on Apache Storm. >> >> For now, improvements are going forward based on current metrics feature. >> >> - Improve (Topology) MetricsConsumer >> <https://issues.apache.org/jira/browse/STORM-1699> >> - Provide topology metrics in detail (metrics per each stream) >> <https://issues.apache.org/jira/browse/STORM-1719> >> - (WIP) Introduce Cluster Metrics Consumer >> >> As I don't maintain large cluster for myself, I really want to collect >> the any ideas for improving, any inconveniences, use cases of Metrics with >> community members, so we're on the right way to go forward. >> >> Let's talk! >> >> Thanks in advance, >> Jungtaek Lim (HeartSaVioR) >> > -- Kevin Conaway http://www.linkedin.com/pub/kevin-conaway/7/107/580/ https://github.com/kevinconaway
