Hi, Jun! > On 16 Aug 2016, at 18:52, Jun Rao <j...@confluent.io> wrote: > > Hi, Andrey, > > For 2, we actually can know the next message size. In LogSegment.read(), we > first use the offset index to find the file position close to the requested > offset and then scan the log forward to find the message whose offset is at > or larger than the requested offset. By the time we find such a message, we > know exactly the size of the first message that we need to return in the > fetch response. No additional read is needed. We can probably just > propagate this information back to the caller. I added a comment related to > this in https://issues.apache.org/jira/browse/KAFKA-3810.
Good point! I think it makes sense to apply the strategy you propose in KAFKA-3810 not only for new fetch request but also for old (“unlimited”) one. So we return full message back even if first unread message exceeds partition limit. And we continue doing so until response limit is reached. What do you think? > > Thanks, > > Jun > Andrey.
