Using Kafka 0.8, when specifying a starting offset and appropriate fetchSize, SimpleConsumer will only return up to the highest offset of the log segment containing the starting offset.
For example, log segment #1 contains offsets 1 - 10 log segment #2 contiains offsets 11 - 100 A fetch request specifying offset 9, will only return offsets 9 and 10. Specifying starting offset 11 will return 11 - 100 Must one check fetchResponse.highwatermark each time to determine if there are higher numbered offsets, and continue submitting fetchRequests ? I assumed SimpleConsumer would return offsets/messages spanning log segments, up to the specified fetchSize. Thanks!
