Hi Neha,
I had to set the "auto.offset.reset" to "smallest" in the particular case where consumer threads weren't running when the data was produced. Then on, its working as you said, ie , as long as the group.id is same, the consumer threads continue to receive new data. Thanks On Mon, Mar 3, 2014 at 10:55 PM, Neha Narkhede <neha.narkh...@gmail.com>wrote: > Question : How to make the consumer threads receive the older data that > was produced when the consumer threads aren't running ? > > As long as you make sure the group.id doesn't change, the consumer will > always > consume data from where it left off during its last run. In other words, > the consumer > can consume data that was produced when it wasn't running, if it is always > started > using the same group.id > > Thanks, > Neha > > > On Mon, Mar 3, 2014 at 8:02 AM, Binita Bharati <binita.bhar...@gmail.com > >wrote: > > > Hi all, > > > > The HighLevelConsumer is working fine. Ref : > > https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example > . > > > > I was expecting the consumer threads to be able to receive older data ( > ie > > the data put into the topic when the consumer threads weren't running ) > > That was the root of the problem, and once, I started my consumer > > application first, and then, generated fresh data into the topic, the > > consumer threads started receiving the new data. > > > > Question : How to make the consumer threads receive the older data that > > was produced when the consumer threads aren't running ? > > > > Thanks > > > > > > On Wed, Feb 26, 2014 at 6:30 PM, Martin Kleppmann > > <mkleppm...@linkedin.com>wrote: > > > > > Hi Binita, > > > > > > The consumer group (group.id) is a mechanism for sharing the load of > > > consuming a high-volume topic between multiple consumers. If you don't > > set > > > a group ID, each consumer consumes all the partitions of a topic. If > you > > > set several consumers to the same group ID, the partitions of the topic > > > you're consuming will be shared amongst the consumers in that group, so > > > that each message is delivered to only one of the consumers in the > group. > > > > > > You can set the group ID to be whatever you like. > > > > > > Hope that helps, > > > Martin > > > > > > On 25 Feb 2014, at 06:27, Binita Bharati <binita.bhar...@gmail.com> > > wrote: > > > > Hi all, > > > > > > > > I am referring to this e.g: > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example > > > . > > > > > > > > What is the consumer group ID being referred here ? > > > > > > > > Thanks > > > > Binita > > > > > > > > >
