Valid assumptions IMO Sent from my iPhone
On Jul 24, 2013, at 8:22 AM, Jay Kreps <jay.kr...@gmail.com> wrote: > Yeah all our monitoring is just of the local process (basically just a > counter exposed through yammer metrics which support jmx and other > outputs). If I understand what you want instead of having a counter that > tracks, say, produce requests per second for a single broker you want one > that covers the whole cluster. Obviously this would require collecting the > local count and aggregating across all the brokers. > > Our assumption is that you already have a separate monitoring system which > can slurp all these up, aggregate them, graph them, and alert off them. > There are a number of open source thingies like this and I think most > bigger shops have something they use. Our assumption is that trying to do a > kafka-specific monitoring system wouldn't work for most people because they > are wedded to their current setup and just want to integrate with that. > > I'm not sure how valid any of those assumptions actually are. > > -Jay > > > On Wed, Jul 24, 2013 at 7:29 AM, Otis Gospodnetic < > otis.gospodne...@gmail.com> wrote: > >> Hi, >> >> I was looking at >> >> https://cwiki.apache.org/confluence/display/KAFKA/Operations#Operations-Monitoring >> and noticed there is no information about which metrics are available >> in which process/JVM/JMX. >> >> Some are available in the Broker process, but some are only available >> from the JVM running Consumer and some only from the JVM running >> Producer. And yet some Producer and Consumer metrics are, I *believe* >> available from Broker's JMX. >> >> Would it be possible for somebody in the know to mark the metrics in >> >> https://cwiki.apache.org/confluence/display/KAFKA/Operations#Operations-Monitoring >> so one can tell where to get it? >> >> Also, why is it that the Broker process doesn't have *all* metrics, >> including Producer and Consumer one? Is that because there can be N >> Brokers and each P or C talk to one Broker at a time and thus there is >> no single process/JMX that can know *all* stats for *all* Brokers and >> for *all* Ps and Cs? >> >> Thank you! >> Otis >> -- >> Performance Monitoring -- http://sematext.com/spm >> Solr & ElasticSearch Support -- http://sematext.com/ >>
