Hmm, It looks as though adding multicast support is more an issue with the Metrics code rather than with kafka-ganglia. The only way I could see to tell GangliaReporter to use multicast when sending messages was to pass it a GangliaMessageBuilder that constructed GangliaMessages using a proper MulticastSocket which sets the multicast TTL.
I attempted to do this like this: https://gist.github.com/ottomata/7269312 However, GangliaMessageBuilder is not a public class, so I can't extend it as MulticastGangliaMessageBuilder. I'm not sure what to do at this point. I'd rather use this library but, it looks like I might have to go with jmxtrans after all. Thoughts? -Andrew On Oct 29, 2013, at 2:38 PM, Maxime Brugidou <maxime.brugi...@gmail.com> wrote: > Hi Andrew, how do you plan to use metrics 3.0.1? The current Kafka 0.8 > version uses 2.2 AFAIK so this is going to require a new Kafka version. > > I don't really use multicast TTL muself but you are right that this should > be configurable and I'll definitely accept pull request going in that > direction. It's a tiny piece of code so it shouldn't be hard to dig it. > > Thanks for using this small plugin and let me know what you plan to do. > > Cheers > On Oct 29, 2013 2:00 PM, "Andrew Otto" <o...@wikimedia.org> wrote: > >> Hi Maxime, >> >> I'm using this at the Wikimedia Foundation to send Kafka Broker metrics to >> Ganglia. However, we use Ganglia in multicast mode. This mostly seems to >> work with your code, but the ttl on the multicast packets gets set at 1. >> We have sometimes have multiple levels of ganglia aggregators, and with >> ttl=1, the multicast packets don't make it to the proper aggregators. >> >> I'm looking into either forking or rewriting this library using Codahale >> Metrics v 3.0.1, and supporting multicast more explicitly. Is this >> something you could do better/faster than me, or should I proceed? :) >> >> -Andrew Otto >> >> (Thanks for writing this, btw!) >> >> >> On Aug 22, 2013, at 11:42 AM, Maxime Brugidou <maxime.brugi...@gmail.com> >> wrote: >> >>> Hi all, >>> >>> Since I couldn't find any other way to publish kafka metrics to ganglia >>> from kafka 0.8 (beta), I just published on github a super-simple ganglia >>> metrics reporter for Kafka. It is configurable through the kafka config >>> file and you can use it on the broker side and on your >> consumers/producers. >>> There is also a feature to exclude some metrics with a regex (useful if >> you >>> have many topics/partions). >>> >>> Here it is on github: https://github.com/criteo/kafka-ganglia >>> >>> Let me know if you have issues/questions. This is using metrics-ganglia >>> 2.2.0 directly so it is not a ganglia plugin but rather a kafka "add-on". >>> >>> I don't know the proper way to distribute this yet for installation on >>> brokers, it could also be part of some contrib kafka code. >>> >>> Cheers, >>> Maxime >>
