Great. Thank you! Fang, Yan [email protected] +1 (206) 849-4108
On Wed, Jul 9, 2014 at 11:45 AM, Tathagata Das <[email protected]> wrote: > 1. Multiple output operations are processed in the order they are defined. > That is because by default each one output operation is processed at a > time. This *can* be parallelized using an undocumented config parameter > "spark.streaming.concurrentJobs" which is by default set to 1. > > 2. Yes, the output operations (and the spark jobs that are involved with > them) gets queued up. > > TD > > > On Wed, Jul 9, 2014 at 11:22 AM, Yan Fang <[email protected]> wrote: > >> I am using the Spark Streaming and have the following two questions: >> >> 1. If more than one output operations are put in the same >> StreamingContext (basically, I mean, I put all the output operations in the >> same class), are they processed one by one as the order they appear in the >> class? Or they are actually processes parallely? >> >> 2. If one DStream takes longer than the interval time, does a new DStream >> wait in the queue until the previous DStream is fully processed? Is there >> any parallelism that may process the two DStream at the same time? >> >> Thank you. >> >> Cheers, >> >> Fang, Yan >> [email protected] >> +1 (206) 849-4108 >> > >
