consumer offset not saved in zk

2013-05-22 Thread rk vishu
Hello All, I recently started experimenting Kafka for my usecase. I am running 0.8 in two node kafka setup. I produced 20messages using a java program(1 partition with 2 replicas) and I am running the consumer code as given in the example https://cwiki.apache.org/KAFKA/consumer-group-example.html

Re: consumer offset not saved in zk

2013-05-23 Thread rk vishu
: > I suspect you had auto.commit.enable=false and consumer.timeout.ms=1. > Can you confirm the values for the above configs in your example? > > Thanks, > Neha > On May 22, 2013 11:22 PM, "rk vishu" wrote: > > > Hello All, > > > > I recently start

Re: consumer offset not saved in zk

2013-05-23 Thread rk vishu
mer > offset is /consumers/[groupId]/offsets/[topic]/[partitionId] -> long > (offset). For more details on our ZK layout, see > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+data+structures+in+Zookeeper > > Thanks, > > Jun > > > On Wed, May 22, 2013 at

Re: consumer offset not saved in zk

2013-05-23 Thread rk vishu
ride the default configs. Also, seems like something > else is wrong with your setup ? Could you share the log4j logs of your > consumer ? Meanwhile, can you try if you can use the console consumer > successfully ? > > Thanks, > Neha > > > On Thu, May 23, 2013 at 2:31 PM, rk

Re: consumer offset not saved in zk

2013-05-23 Thread rk vishu
Neha Narkhede wrote: > Can you please try the following - > > ./sbt clean assembly-package-dependency package > > Thanks, > Neha > > > On Thu, May 23, 2013 at 3:16 PM, rk vishu wrote: > > > Neha, > > > > Thanks for pointing out the log4j. I turned

Recommendation for number of brokers on kafka(0.7.2) hosts

2013-10-01 Thread rk vishu
Hello All, I am currently using 5 node kafka cluster with 0.7.2 version. Would like to get some advice on optimal number of brokers on each kafka host. Below is the specification of each machine - 4 data directories /data1,/data2, /data3, /data4 with 200+GB usable space. RAID10 - 24 Core CPU - 32

Re: Recommendation for number of brokers on kafka(0.7.2) hosts

2013-10-01 Thread rk vishu
s through the log.dirs > config. > > Thanks, > Neha > > > On Tue, Oct 1, 2013 at 1:17 PM, rk vishu wrote: > > > Hello All, > > > > I am currently using 5 node kafka cluster with 0.7.2 version. Would like > to > > get some advice on optimal