Here’s an article that Gwen wrote earlier this year on handling large messages in Kafka.
http://ingest.tips/2015/01/21/handling-large-messages-kafka/ -James > On Oct 5, 2015, at 11:20 AM, Pradeep Gollakota <pradeep...@gmail.com> wrote: > > Fellow Kafkaers, > > We have a pretty heavyweight legacy event logging system for batch > processing. We're now sending the events into Kafka now for realtime > analytics. But we have some pretty large messages (> 40 MB). > > I'm wondering if any of you have use cases where you have to send large > messages to Kafka and how you're dealing with them. > > Thanks, > Pradeep ________________________________ This email and any attachments may contain confidential and privileged material for the sole use of the intended recipient. Any review, copying, or distribution of this email (or any attachments) by others is prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete this email and any attachments. No employee or agent of TiVo Inc. is authorized to conclude any binding agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo Inc. may only be made by a signed written agreement.
