[
https://issues.apache.org/jira/browse/KAFKA-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14056792#comment-14056792
]
Julian Morrison commented on KAFKA-1532:
----------------------------------------
Is "Kafka itself fully materializes messages in memory" unavoidable? (I am
assuming writing it to file buffers is obvious.)
> Move CRC32 to AFTER the payload
> -------------------------------
>
> Key: KAFKA-1532
> URL: https://issues.apache.org/jira/browse/KAFKA-1532
> Project: Kafka
> Issue Type: Improvement
> Components: core, producer
> Reporter: Julian Morrison
> Assignee: Jun Rao
> Priority: Minor
>
> To support streaming a message of known length but unknown content, take the
> CRC32 out of the message header and make it a message trailer. Then client
> libraries can calculate it after streaming the message to Kafka, without
> materializing the whole message in RAM.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
