Re: Kafka 0.8 jmxtrans + puppet

2013-11-06 Thread Andrew Otto
We will be using Kafka to transport access logs from frontend cache servers to HDFS. But! We aren't doing this in production yet, so I wouldn't quite yet feel comfortable saying we are powered by Kafka. I hope to get this into production more in a few weeks*, and at that time we'll write a ni

Re: Kafka 0.8 jmxtrans + puppet

2013-11-06 Thread Joe Stein
Yes, thank you Andrew this is great stuff. May I ask how Wikimedia is using Kafka so I can update the powered by https://cwiki.apache.org/confluence/display/KAFKA/Powered+By /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC

Re: Kafka 0.8 jmxtrans + puppet

2013-11-06 Thread Neha Narkhede
Cool, thanks for sharing this! -Neha On Nov 6, 2013 6:16 AM, "Andrew Otto" wrote: > Hi, > > I just got jmxtrans set up with Kafka 0.8 over at Wikimedia. We're using > puppet, and the https://github.com/wikimedia/puppet-kafka module to > install and set up Kafka. This module now comes with a >

Kafka 0.8 jmxtrans + puppet

2013-11-06 Thread Andrew Otto
Hi, I just got jmxtrans set up with Kafka 0.8 over at Wikimedia. We're using puppet, and the https://github.com/wikimedia/puppet-kafka module to install and set up Kafka. This module now comes with a kafka::server::jmxtrans class which will automatically set up jmxtrans with some nice Kafka q