If your developing in Scala then check out my Scala Example code repository https://github.com/stealthly/scala-kafka. It spins up two virtual machines (zookeeper and a kafka broker) and has a sample producer and consumer with passing test cases (against the vms).
If your developing in Java, the Vagrantfile and scripts folder should still work fine for you and you can fork the repo and include whatever language you want on top of it and use the virtual zookeeper and kafka broker parts of it. Feedback welcome, thanks! /******************************************* Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> ********************************************/ On Thu, Dec 19, 2013 at 12:18 AM, Jun Rao <jun...@gmail.com> wrote: > You can take a look at Kafka's unit tests. See how > kafka.integration.KafkaServerTestHarness is being used. > > Thanks, > > Jun > > > On Wed, Dec 18, 2013 at 11:17 AM, Oliver Dain <od...@3cinteractive.com > >wrote: > > > Found some of the other references I'd used that also don't quite work: > > > > https://gist.github.com/fjavieralba/7930018 > > https://gist.github.com/mardambey/2650743 > > > > I know, the obvious question is what didn't work. I'm afraid I no longer > > recall. I took each example, tried it, got errors, tried to fix them > (e.g. > > change properties names), etc. but it was on a deadline and several days > > ago and I no longer recall exactly what I did. I can try to recreate if > > people think one of those should work, but I suspect there have been > known > > changes. > > > > It would be fantastic if this was all wrapped up in a single method or > > class so creating an embedded broker and/or cluster for testing was > really > > easy. > > > > > > From: New User <od...@3cinteractive.com<mailto:od...@3cinteractive.com>> > > Date: Wednesday, December 18, 2013 at 11:10 AM > > To: "users@kafka.apache.org<mailto:users@kafka.apache.org>" < > > users@kafka.apache.org<mailto:users@kafka.apache.org>> > > Subject: Writing unit tests for Kafka code > > > > I'm writing some Kafka client code and I'd like to unit test it. I've > > found some resources that say this is possible. For example: > > > > http://ransilberman.wordpress.com/2013/07/19/how-to-unit-test-kafka/ > > > > but all the information I've found seems a little bit incorrect. For > > example, the above link uses "enable.zookeeper" which seems to be an > > ignored property. Similarly, other properties mentioned in that file have > > changed names. I found a few posts on this mailing list describing how it > > might be done (sorry - this was a few days ago and I don't have links > > handy), but they didn't quite work either. I tried digging into the scala > > code but it wasn't immediately obvious how to do things, probably > partially > > because I don't know Scala. > > > > If somebody could post working Java code that constructs a Kafka broker > > that can be used for unit testing that would be fantastic. My preference > > would be something that doesn't require ZooKeeper, but there's some talk > > that in 0.8 that may no longer be an option. If not, using the Curator > > framework's TestingServer would be great (I'm using Curator everyone). > But > > ultimately, I'll take anything that works! > > > > Thanks in advance, > > Oliver > > >
