Why not use tools like jmxtrans <https://github.com/jmxtrans/jmxtrans/wiki> and send your metrics to Graphite/OpenTsdb.etc? Why do serialization/de-serialization twice?
On Fri, Apr 1, 2016 at 9:51 AM, Gerard Klijs <gerard.kl...@dizzit.com> wrote: > Don't know if adding it to Kafka is a good thing. I assume you need some > java opts settings for it to work, and with other solutions these would be > different. It could be enabled with an option off course, then it's not in > the way if you use something else. > We use zabbix, this is a single tool which can be used to read in jmx data, > store the data for a certain time, configurable for each item, and create > triggers and graphs for those. > > To see and copy jmx items we use the Oracle Java mission control, it has a > tab with info on each jmx item, which can be copied to clipboard. > > On Fri, Apr 1, 2016, 02:03 Sean Clemmer <sclem...@bluejeansnet.com> wrote: > > > Another +1 for Jolokia. We've got a pretty cool setup here that deploys > > Jolokia alongside Kafka, and we wrote a small Sensu plugin to grab all > the > > stats from Jolokia's JSON API and reformat them for Graphite. > > > > On Thu, Mar 31, 2016 at 4:36 PM, craig w <codecr...@gmail.com> wrote: > > > > > Including jolokia would be great, I've used for kafka and it worked > well. > > > On Mar 31, 2016 6:54 PM, "Christian Posta" <christian.po...@gmail.com> > > > wrote: > > > > > > > What if we added something like this to Kafka? https://jolokia.org > > > > I've added a JIRA to do that, just haven't gotten to it yet. Will > soon > > > > though, especially if it'd be useful for others. > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-3377 > > > > > > > > On Thu, Mar 31, 2016 at 2:55 PM, David Sidlo <dsi...@ancestry.com> > > > wrote: > > > > > > > > > The Kafka JmxTool works fine although it is not user friendly, in > > that > > > > you > > > > > cannot perform a query of the Kafka Server mbeans to determine > > content > > > > and > > > > > to determine the path-string that you need to place into the > > > -object-name > > > > > option. > > > > > > > > > > Here's how I solved the problem... > > > > > > > > > > First, make sure that Kafka is running with jms options enabled... > > > > > > > > > > - The following opens up the jxm port with no > authentication > > > > (for > > > > > testing)... > > > > > -Dcom.sun.management.jmxremote > > -Dcom.sun.management.jmxremote.port=9999 > > > > > -Dcom.sun.management.jmxremote.ssl=false > > > > > -Dcom.sun.management.jmxremote.authenticate=false > > > > > > > > > > Second, get jstatd running on the same server so that you can use > > > > VisualVM > > > > > to look into what is going on inside. > > > > > > > > > > Then, use VisualVM along with its jmx-plugin to view the mbeans and > > > > > contents. > > > > > > > > > > When using VisualVM, you will first connect to jstatd, then you > have > > to > > > > > right click on the host to request a JMX connection, where you will > > > get a > > > > > dialog, where you have to add the jmx port number to the host name > > > (after > > > > > the colon). > > > > > > > > > > When you view the mbeans, you will see the path to the given > > attribute > > > if > > > > > you hover over it with the mouse pointer, but only for a moment... > > The > > > > > string can be quite long and... Unfortunately, you don't have the > > > option > > > > to > > > > > capture that string into the cut-buffer via the interface. I used a > > > > screen > > > > > capture utility to capture the string and typed it into the > terminal. > > > > > > > > > > So here is what my first working query looked like... > > > > > > > > > > /opt/kafka/kafka/bin/kafka-run-class.sh kafka.tools.JmxTool > > > --object-name > > > > > > > > > > > > > > > "kafka.consumer:type=FetchRequestAndResponseMetrics,name=FetchRequestRateAndTimeMs,clientId=ReplicaFetcherThread-0-5,brokerHost= > > > > > hostname05.cluster.com,brokerPort=9092" --jmx-url > > > > > service:jmx:rmi:///jndi/rmi://`hostname`:9999/jmxrmi > > > > > > > > > > That command will output all of the attributes for the given > > > object-name, > > > > > and it's a long object-name. > > > > > With some experimentation, I found that you can use wild-cards on > > > > portions > > > > > of the object-name that were clearly dynamic, such as the host-name > > or > > > > the > > > > > thread-name. So you can get the attribute for all similar objects > by > > > > using > > > > > the following query... > > > > > > > > > > /opt/kafka/kafka/bin/kafka-run-class.sh kafka.tools.JmxTool > > > --object-name > > > > > > > > > > > > > > > "kafka.consumer:type=FetchRequestAndResponseMetrics,name=FetchRequestRateAndTimeMs,clientId=ReplicaFetcherThread*,brokerHost=hostname*. > > > > > cluster.com,brokerPort=*" --jmx-url > > > > > service:jmx:rmi:///jndi/rmi://`hostname`:9999/jmxrmi > > > > > > > > > > There may be simpler tools to use, but I really do like the GUI > > > goodness > > > > > in VisualVM, (and it is free). > > > > > > > > > > I hope that helps. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > *Christian Posta* > > > > twitter: @christianposta > > > > http://www.christianposta.com/blog > > > > http://fabric8.io > > > > > > > > > > -- Regards Vamsi Subhash
