Re: count of bytes stored in kafka

2013-12-27 Thread Barto Rael
Hi Otis, Thanks for response! We are using kafka 0.8.0. I was looking for some jmx metrix that can give count of messages/bytes stored in kafka. If we look into data stores like hbase we can see how many messages are stored in hbase or if we look into search engine like elasticsearch then also

Re: Writing unit tests for Kafka code

2013-12-27 Thread Oliver Dain
I finally got something working. It was a bit funky, so I figure I¹ll post it here in hopes that it will help someone else. First, you need to include the test stuff from Kafka. If using Maven, this does the trick: org.apache.kafka kafka_2.10 test test I used Apache Curator to get

Re: count of bytes stored in kafka

2013-12-27 Thread Otis Gospodnetic
Hi, Not a direct way to see this, but we use Kafka 0.7.2 and monitor the disk space on it, so that's one way to see *roughly* what is happening. If you use something like SPM to monitor your Kafka nodes (see http://blog.sematext.com/2013/10/16/announcement-spm-performance-monitoring-for-kafka/) an

Re: Writing unit tests for Kafka code

2013-12-27 Thread Oliver Dain
Hey Joe, Thanks for the suggestion. The code looks pretty cool, but it¹s much heavier weight than what I¹m looking for. I can spin up ZooKeeper and Kafka on my local box, but I was hoping to find a way to run them embedded so I can create unit tests (as opposed to integration tests which would use

Re: Writing unit tests for Kafka code

2013-12-27 Thread Oliver Dain
Hi Jun, Thanks for getting back to me. Unfortunately, I can't seem to get KafkaServerTestHarness to work for me. I've looked around the internet for examples but can't find any in Java. My Java compiler seems to think that KafkaServerTestHarness isn't a class so it won't let me instantiate one

Re: node.js client library?

2013-12-27 Thread Joe Stein
I added the wurstmeister client to the wiki SOHU-Co, can you provide a license file in the project and I would link it then too please. https://cwiki.apache.org/confluence/display/KAFKA/Clients I also added wurstmeister's port of storm-kafka for 0.8.0 also to the client list and my company's Sca