Any thoughts on upgrading from 0.7 to 0.8 you'd like to share with the community? How did it go? How has it been?
Philip ----------------------------------------- http://www.philipotoole.com On Wednesday, August 20, 2014 10:13 AM, Cory Watson <gp...@keen.io> wrote: :P Thanks Philip! On Wed, Aug 20, 2014 at 10:04 AM, Philip O'Toole <philip.oto...@yahoo.com.invalid> wrote: Nice work. That tool I put together was getting a bit old. :-) > > >I updated the Kafka "ecosystem" page with details of both tools. > >https://cwiki.apache.org/confluence/display/KAFKA/Ecosystem > >Philip > > > >----------------------------------------- >http://www.philipotoole.com > > > >On Wednesday, August 20, 2014 9:32 AM, Cory Watson <gp...@keen.io> wrote: > > > >Hello all! > >If you use Storm and Kafka 0.8 together I may have an interesting project >for you. Recently at Keen IO <https://keen.io/> we upgraded from Kafka 0.7 > >to 0.8 and needed to replace the features of stormkafkamon ><https://github.com/otoolep/stormkafkamon> for monitoring spout offsets > >since it only worked with Kafka 0.7. So we created Capillary ><https://github.com/keenlabs/capillary>! > > >Given a bit of information ><https://github.com/keenlabs/capillary#configuration> about your Zookeeper > >setup for Storm and Kafka, Capillary is a web application that >automatically discovers topologies running Kafka 0.8 spouts and lets you >click on them to see per-partition offsets ><https://raw.githubusercontent.com/keenlabs/capillary/master/shot.png>. > >It's JVM based and shouldn't require anything but Java where you run it. We >decided this was a useful difference over the >install-on-each-host-with-virtualenv model used by stormkafkamon. > >In addition to the helpful UI there is an API that you might use for >collecting this information and sending it to your stats/observability >systems for historical or alerting purposes. There's even an example python >script ><https://github.com/keenlabs/capillary/blob/master/stats-to-datadog.py> we >use for reporting to Datadog <http://www.datadoghq.com/>. It's very likely > >we'll move this to a scheduled thread inside Capillary using metrics ><http://metrics.codahale.com/> and allow you to hook up your own reporter. > > >I hope other uses of Kafka + Storm find this useful. Feel free to open an >issue if you have issues setting it up or configuring it, or drop PRs if >you have additional functionality. Thanks! > >(I'll be cross-posting this to the Storm list also!) > >-- >Cory Watson >Principal Infrastructure Engineer // Keen IO -- Cory Watson Principal Infrastructure Engineer // Keen IO
