Re: consumer offset not saved in zk

2013-05-23 Thread rk vishu
Neha, I see the point. I verified zkclient version from kafka build and found that it is 0.2. i updated my client app's POM to include the following (corrected from 0.1) com.101tec zkclient 0.2 Thank you very much for the inputs and help. On Thu, May 23, 2013 at 3:23 PM, Neha N

Re: consumer offset not saved in zk

2013-05-23 Thread Neha Narkhede
Can you please try the following - ./sbt clean assembly-package-dependency package Thanks, Neha On Thu, May 23, 2013 at 3:16 PM, rk vishu wrote: > Neha, > > Thanks for pointing out the log4j. I turned on logs at INFO level. Now i > see some warnings as below. > > WARN [Kafka-consumer-autocom

Re: consumer offset not saved in zk

2013-05-23 Thread rk vishu
Neha, Thanks for pointing out the log4j. I turned on logs at INFO level. Now i see some warnings as below. WARN [Kafka-consumer-autocommit-1] (Logging.scala:88) - [1_BELC02K41GGDKQ4.sea.corp.expecn.com-1369346576173-22148419], exception during commitOffsets java.lang.NoSuchMethodError: org.I0Itec

Re: consumer offset not saved in zk

2013-05-23 Thread Neha Narkhede
You don't want to override the default configs. Also, seems like something else is wrong with your setup ? Could you share the log4j logs of your consumer ? Meanwhile, can you try if you can use the console consumer successfully ? Thanks, Neha On Thu, May 23, 2013 at 2:31 PM, rk vishu wrote: >

Re: consumer offset not saved in zk

2013-05-23 Thread rk vishu
My ZK directory listing is as below. Looks like offsets path is not even created. zk: localhost:2181(CONNECTED) 0] ls / [hadoop-ha, hbase, zookeeper, consumers, controller, storm, brokers, controller_epoch] [zk: localhost:2181(CONNECTED) 1] ls /consumers [1, das-service] [zk: localhost:2181(CONNEC

Re: consumer offset not saved in zk

2013-05-23 Thread rk vishu
Neha, below are my properties. I tried adding consumer.timeout.ms=3000 or 1 also. Properties props = new Properties(); props.put("zookeeper.connect", a_zookeeper); props.put("group.id", "1"); props.put("zookeeper.session.timeout.ms", "4000"); props.put("zookee

Re: consumer offset not saved in zk

2013-05-23 Thread Jun Rao
You are looking at the wrong path in ZK. The correct path for consumer offset is /consumers/[groupId]/offsets/[topic]/[partitionId] -> long (offset). For more details on our ZK layout, see https://cwiki.apache.org/confluence/display/KAFKA/Kafka+data+structures+in+Zookeeper Thanks, Jun On Wed, M

Re: consumer offset not saved in zk

2013-05-23 Thread Neha Narkhede
I suspect you had auto.commit.enable=false and consumer.timeout.ms=1. Can you confirm the values for the above configs in your example? Thanks, Neha On May 22, 2013 11:22 PM, "rk vishu" wrote: > Hello All, > > I recently started experimenting Kafka for my usecase. I am running 0.8 in > two n

consumer offset not saved in zk

2013-05-22 Thread rk vishu
Hello All, I recently started experimenting Kafka for my usecase. I am running 0.8 in two node kafka setup. I produced 20messages using a java program(1 partition with 2 replicas) and I am running the consumer code as given in the example https://cwiki.apache.org/KAFKA/consumer-group-example.html