Consumer's Group.id and autooffset.reset

2014-05-05 Thread Chris Helck
Would some one explain the interplay between the group.id and autooffset.reset config parameters? I wrote a simple Consumer that had group.id = "XYZZY" autooffset.reset = smallest My intention was to write something similar to Kafka's ConsoleConsumer; that each time my Consumer ran, it

RE: Need example of robust Producer

2014-05-01 Thread Chris Helck
@kafka.apache.org Subject: Re: Need example of robust Producer Does this help: https://github.com/stealthly/scala-kafka/blob/master/src/main/scala/KafkaPr oducer.scala -Dragos On 4/30/14, 5:29 AM, "Chris Helck" wrote: >I am new to Kafka. I'd like to see example code that uses a Pro

Need example of robust Producer

2014-04-30 Thread Chris Helck
I am new to Kafka. I'd like to see example code that uses a Producer and handles connectivity errors to Kafka brokers and/or ZooKeeper. I have seen some code that creates and tears down a Producer for each message that is sent, but this doesn't seem right to me. The same example code, queries Z

New to Kafka: ZooKeeper and Client

2014-04-29 Thread Chris Helck
I have a few newbie questions. I need to create a Producer that sends messages to Kafka brokers. Does a machine running a Kafka client (Producer) need its own instance of ZooKeeper running? Or does is simply connect to the ZooKeeper that is running remotely with the Kafka brokers?