Re: Samza hdfs

2015-11-13 Thread Rick Mangi
We also evaluated Spark Streaming as well as Storm before settling on Samza for our work. Our reason was the confidence and modularity that comes with having Kafka between each job in the topology. Being able to treat each job as a separate deployment makes ongoing development much easier and w

Re: Samza hdfs

2015-11-13 Thread Aram Mkrtchyan
Hi Navina, Sure , I will do it with great pleasure. And a little about why we have moved from Spark Streaming to Samza : The main reason was the job scheduling on each batch processing. Also, we need to process messages one by one, which means whenever the message is consumed it needs to be p

Re: Samza hdfs

2015-11-11 Thread Navina Ramesh
Hi Aram, I think a Cassandra producer will definitely be useful for some of the Samza users. It will be great if you can contribute back to Samza! I am also curious to understand what motivated you to move off from Spark Streaming. Thanks! Navina On Tue, Nov 10, 2015 at 11:21 PM, Bae, Jae Hyeon

Re: Samza hdfs

2015-11-11 Thread Aram Mkrtchyan
Hi Jae, The main reason was the job scheduling on each batch processing. Also, we need to process messages one by one, which means whenever the message is consumed it needs to be processed immediately. Thanks. On Wed, Nov 11, 2015 at 11:21 AM, Bae, Jae Hyeon wrote: > Hi Aram > > May I know wh

Re: Samza hdfs

2015-11-10 Thread Bae, Jae Hyeon
Hi Aram May I know why you're moving off from Spark streaming to Samza? On Tue, Nov 10, 2015 at 11:04 PM, Aram Mkrtchyan < aram.mkrtch...@picsart.com.invalid> wrote: > Hi Yi, > Many thanks! :) > It's wonderful ! We really need samza-hdfs module. > Thanks to samza being very simple we have been a

Re: Samza hdfs

2015-11-10 Thread Aram Mkrtchyan
Hi Yi, Many thanks! :) It's wonderful ! We really need samza-hdfs module. Thanks to samza being very simple we have been able to create cassandra producer for samza and we can share it if you think it's will be useful for others. Thanks. On Wed, Nov 11, 2015 at 9:13 AM, Yi Pan wrote: > Hi, Aram

Re: Samza hdfs

2015-11-10 Thread Yi Pan
Hi, Aram, Welcome to the community! :) We are planning to release 0.10 in this month and YES HDFS producer is included in this release! -Yi On Tue, Nov 10, 2015 at 3:18 AM, Aram Mkrtchyan < aram.mkrtch...@picsart.com.invalid> wrote: > Hi guys, > > We are considering to move from spark streaming