sambhav-jain-16 commented on PR #13646: URL: https://github.com/apache/kafka/pull/13646#issuecomment-1547640800
I was able to know the reason why the position reported by `consumer.position()` is more than the actual records consumed. Transactional producer produces commit messages into the user topic itself which is marked as a "Control" batch. However when these messages are consumed, the consumer deliberately skips these messages and doesn't add them to the returned `records` list. However, the offsets are moved forward to continue for further batches. https://github.com/apache/kafka/blob/fa7818dff5a28048401654a7497e56dbc988b755/clients/src/main/java/org/apache/kafka/clients/consumer/internals/CompletedFetch.java#L214-L222 Now the issue with `consumeAll` method is that it exits when it reaches particular offsets rather than consuming certain number of messages which `consume` does. I think if our use-case of this test is to consume minimum messages rather than offsets, we should use `consume`. However, if the intention of `consumeAll` is to do something in the same lines (which looks like from the java doc of the method), we should change the `consumeAll` method. @C0urante can you comment on this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
