Re: Kafka offset using kafka topic - not consuming messages

2015-04-29 Thread Gomathivinayagam Muthuvinayagam
Thank you, I am using the same groupId all the time. I printed OffsetsMessageFormatter output for the consumer, and the output is showing as [async_force_consumers,force_msgs,9]::OffsetAndMetadata[2,associated metadata,1430277791077] But If I restart the consumer, it starts consuming messages fr

Re: Kafka offset using kafka topic - not consuming messages

2015-04-29 Thread Jiangjie Qin
OK, so you turned off auto.offset.commit, and set the auto.offset.reset to largest. That means when you consume, 1. If you did not commit offsets manually, no offsets will be committed to Kafka. 2. If you do not have an offset stored in Kafka, you will start from the log end and ignore the existin

Kafka offset using kafka topic - not consuming messages

2015-04-29 Thread Gomathivinayagam Muthuvinayagam
I am using Kafka 0.8.2 and I am using Kafka based storage for offset. Whenever I restart a consumer (high level consumer api) it is not consuming messages whichever were posted when the consumer was down. I am using the following consumer properties Properties props = new Properties();