When you iterate the fetch result, MessageAndOffset has a nextOffet() method. You need to use that offset to initiate the next fetch request. See https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Examplefor an example.
Thanks, Jun On Fri, Jun 28, 2013 at 9:53 PM, Ran RanUser <ranuse...@gmail.com> wrote: > Using Kafka 0.8, when specifying a starting offset and appropriate > fetchSize, SimpleConsumer will only return up to the highest offset of the > log segment containing the starting offset. > > For example, > > log segment #1 contains offsets 1 - 10 > log segment #2 contiains offsets 11 - 100 > > A fetch request specifying offset 9, will only return offsets 9 and 10. > Specifying starting offset 11 will return 11 - 100 > > Must one check fetchResponse.highwatermark each time to determine if there > are higher numbered offsets, and continue submitting fetchRequests ? > > I assumed SimpleConsumer would return offsets/messages spanning log > segments, up to the specified fetchSize. > > > Thanks! >
