[
https://issues.apache.org/jira/browse/KAFKA-3442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205631#comment-15205631
]
Jun Rao commented on KAFKA-3442:
--------------------------------
[~becket_qin], yes, the 0.8.2 consumer is able to consume the large message. It
seems that the issue is the following. For fetch request of v0/v1, we need to
call FileMessageSet.toMessageFormat() to do the down conversion. In that
method, we call FileMessageSet.iterator() to iterate all messages. However, the
iterator seems to have an bug that it only checks "end" at the beginning of
each message iteration, but not when it reads size and the actual payload of
the message. Because of this, the iterator can return a message that passes
"end". That message will then be returned to the consumer after down conversion.
> FetchResponse size exceeds max.partition.fetch.bytes
> ----------------------------------------------------
>
> Key: KAFKA-3442
> URL: https://issues.apache.org/jira/browse/KAFKA-3442
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.10.0.0
> Reporter: Dana Powers
> Priority: Blocker
> Fix For: 0.10.0.0
>
>
> Produce 10000 byte message to topic foobar
> Fetch foobar w/ max.partition.fetch.bytes=1024
> Test expects to receive a truncated message (~1024 bytes). 0.8 and 0.9 pass
> this test, but 0.10 FetchResponse has full message, exceeding the max
> specified in the FetchRequest.
> I tested with v0 and v1 apis, both fail. Have not tested w/ v2
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
