The fetch/message protocol described here <https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-FetchResponse> shows that the MessageSet returned from a FetchRequest should be structured as:
MessageSet => [Offset MessageSize Message] Offset => int64 MessageSize => int32 However, it looks like the Kafka broker can actually return an incomplete message for the last message in the set due to the size set in MaxBytes. In the returned MessageSet the last MessageSize returned is larger than the total remaining bytes. It looks like an incomplete value can be returned for any part of the message set (Offset and MessageSize) too, though I have not confirmed. This means the MessageSize can't be trusted and the protocol is not actually correct. Is this expected behavior? If expected, is it good behavior? -- Grant Henke Solutions Consultant | Cloudera ghe...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
