Re: Simple Question: Spark Streaming Applications

2014-09-30 Thread Tathagata Das
More importantly, why are you asking this question? :) Also let me generalize the answer by saying that most applications that do some useful computations use map-like operations. And by map-like operations I mean simple operations like map, filter, flatMap, mapPartitions. The only category of appl

Re: Simple Question: Spark Streaming Applications

2014-09-30 Thread Tobias Pfeiffer
Hi, On Wed, Oct 1, 2014 at 12:20 AM, Saiph Kappa wrote: > But most applications use transformations, and map in particular, correct? > Yes, I would claim that most applications that do some useful computation use map(). Tobias

Re: Simple Question: Spark Streaming Applications

2014-09-30 Thread Saiph Kappa
But most applications use transformations, and map in particular, correct? On Mon, Sep 29, 2014 at 6:21 PM, Liquan Pei wrote: > Hi Saiph, > > Map is used for transformation on your input RDD. If you don't need > transformation of your input, you don't need to use map. > > Thanks, > Liquan > > O

Re: Simple Question: Spark Streaming Applications

2014-09-29 Thread Liquan Pei
Hi Saiph, Map is used for transformation on your input RDD. If you don't need transformation of your input, you don't need to use map. Thanks, Liquan On Mon, Sep 29, 2014 at 10:15 AM, Saiph Kappa wrote: > Hi, > > Do all spark streaming applications use the map operation? or the majority > of

Simple Question: Spark Streaming Applications

2014-09-29 Thread Saiph Kappa
Hi, Do all spark streaming applications use the map operation? or the majority of them? Thanks.