Re: Samza for text processing

2019-04-30 Thread Jagadish Venkatraman
Hi Rob, *>> We are currently deciding between kafka streams and* *Samza. Which do you think would be more appropriate?* Roughly, the two are similar - The design of Samza certainly influenced what went into Kafka Streams. However, here are some key differences: - Support for non-Kafka source an

Re: Samza for text processing

2019-04-28 Thread Rob Martin
Thanks for the reply. We are currently deciding between kafka streams and Samza. Which do you think would be more appropriate? Also for files over 1mb would you increase the default kafka limit? Break the document into chunks or pass a reference in the message? Thanks again On Sun, 28 Apr 2019

Re: Samza for text processing

2019-04-28 Thread Jagadish Venkatraman
Hi Rob, Yes, your use-case is a good fit. You can use Samza for fault-tolerant stream processing. We have document (eg: member profiles, articles/blogs) standardization use-cases at LinkedIn powered by Samza. Please let us know should you have further questions! On Sun, Apr 28, 2019 at 7:09 AM