Hi, I don't have any source or configs handy to check things, but you are saying you've configured Kafka to use GraphiteReporter, right? So why not remove that config, so metrics stop being sent to Graphite if your Graphite setup is suffering? If you do that and you still want to see your Kafka metrics, you can always use SPM <http://sematext.com/spm/> for Kafka (though some of the graphs will be empty until we get KAFKA-1481, or something else that improves metrics, in). If you just want to use it temporarily, just use the free 30-day trial version until you beef up your Graphite setup.
Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Elasticsearch Support * http://sematext.com/ On Fri, Sep 19, 2014 at 10:08 AM, François Langelier <f.langel...@gmail.com> wrote: > Hi Daniel, > > Thank you for your answer. > > It's possible that I didn't understood something, if so correct me please. > > From what I understood, from the kafka doc #monitoring > <http://kafka.apache.org/documentation.html#monitoring>, kafka use Yammer > Metrics for monitoring the servers (the brokers) and the clients (producers > and consumers). > > Our web site is also using Yammer Metrics and push that to our Graphite > server and our web site also produce message in kafka. > From what I read, the Yammer Metrics GraphiteReporter is a kind of > Singleton, once I Enable it, it is working for all the process. (But I > might be wrong here...) > > We recently upgrade kafka from 0.7.2 to 0.8.1.1 and since the upgrade, > kafka is monitoring in our Graphite Server and is hammering it, so we > aren't able to use it because we always get timeout... > > SO, I was wondering if there is a way to disable the kafka monitoring to > our Graphite server. > > We are using the code in the tag 0.8.1.1 on github, so if the kafka-ganglia > isn't in the tag, we aren't using it :) > > > François Langelier > Étudiant en génie Logiciel - École de Technologie Supérieure > <http://www.etsmtl.ca/> > Capitaine Club Capra <http://capra.etsmtl.ca/> > VP-Communication - CS Games <http://csgames.org> 2014 > Jeux de Génie <http://www.jdgets.com/> 2011 à 2014 > Magistrat Fraternité du Piranha <http://fraternitedupiranha.com/> > Comité Organisateur Olympiades ÉTS 2012 > Compétition Québécoise d'Ingénierie 2012 - Compétition Senior > > On Wed, Sep 17, 2014 at 11:05 PM, Daniel Compton <d...@danielcompton.net> > wrote: > > > Hi Francois > > > > I didn't quite understand how you've set up your metrics reporting. Are > you > > using the https://github.com/criteo/kafka-ganglia metrics reporter? If > so > > then you should be able to adjust the config to exclude the metrics you > > don't want, with kafka.ganglia.metrics.exclude.regex. > > > > > > On 18 September 2014 07:55, François Langelier <f.langel...@gmail.com> > > wrote: > > > > > Hi all! > > > > > > We are using yammer metrics to monitor some parts of our system. > > > > > > Since we upgrade from kafka 0.7.2 to 0.8.1.1, we saw a lot more data > > > getting in our graphite server and from what I saw, it looks like it > all > > > come from our producers. > > > > > > From what i understand, since we already use graphite, our > > graphiteReporter > > > is enable in our main web site and our kafka producers are having fun > > using > > > it too to monitor in graphite. > > > > > > The problem is that right now kafka is hammering of graphite server and > > we > > > have difficulty to saw our monitored data... > > > > > > Is there a way to deactivate the monitoring of our kafka producers? > > > > > > > > > François Langelier > > > Étudiant en génie Logiciel - École de Technologie Supérieure > > > <http://www.etsmtl.ca/> > > > Capitaine Club Capra <http://capra.etsmtl.ca/> > > > VP-Communication - CS Games <http://csgames.org> 2014 > > > Jeux de Génie <http://www.jdgets.com/> 2011 à 2014 > > > Magistrat Fraternité du Piranha <http://fraternitedupiranha.com/> > > > Comité Organisateur Olympiades ÉTS 2012 > > > Compétition Québécoise d'Ingénierie 2012 - Compétition Senior > > > > > >
