Hi! I did something similar. You can use the high level consumer but turn off auto commit and commit only what you are done with. Here's the code I used: https://github.com/iamnoah/kakfa-offsets-test
On Thu, Jul 9, 2015 at 4:53 PM Shashank Singh <shashank.ru...@gmail.com> wrote: > Hi Team > > I was going over the documentation and links related to consumer API > provided in > 0.8.2.1 release > I got to know of major improvement in consumer API in 0.8.3 release. May I > know when we are planning to have this release. > > One more help - I was following up the example given on > https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example > > In this approach we can't decide > a) how to get the current offset > b) how to set a particular consumer group to start reading from a specific > offset > > Also how to use the same example to read one message at a time from a topic > with many topics. I tried with creating a list of iterators and then > checking for hasNext() but that call in blocking. Even though I am using a > timeout on that call this seems not be working. > > Instead what I have settled with is an internal queue to which the > multi-threaded consumer offers the messages into. Poll API polls the > internal queue. Problem is in case client code gets down without consuming > all the messages the offset is already set and the messages which have > already been read from iterator and put in queue will be never seen again. > > > -- > > *Warm Regards,* > > *Shashank * > > *Mobile: +91 9910478553 * > > *Linkedin: in.linkedin.com/pub/shashank-singh/13/763/906/ > <http://in.linkedin.com/pub/shashank-singh/13/763/906/>* >
