[
https://issues.apache.org/jira/browse/KAFKA-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14056444#comment-14056444
]
Jay Kreps commented on KAFKA-1532:
----------------------------------
Also since Kafka itself fully materializes messages in memory, fixing this
won't actually let clients send arbitrarily large messages, as the broker will
still choke.
> Move CRC32 to AFTER the payload
> -------------------------------
>
> Key: KAFKA-1532
> URL: https://issues.apache.org/jira/browse/KAFKA-1532
> Project: Kafka
> Issue Type: Improvement
> Components: core, producer
> Reporter: Julian Morrison
> Assignee: Jun Rao
> Priority: Minor
>
> To support streaming a message of known length but unknown content, take the
> CRC32 out of the message header and make it a message trailer. Then client
> libraries can calculate it after streaming the message to Kafka, without
> materializing the whole message in RAM.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
