Re: Spark Streaming Threading Model

2014-12-19 Thread jay vyas
So , at any point does a stream stop producing RDDs ? If not, is there a possibility, if the batching isnt working or is broken, that your disk / RAM will fill up to the brim w/ unprocessed RDD backlog? On Fri, Dec 19, 2014 at 1:29 PM, Silvio Fiorito < silvio.fior...@granturing.com> wrote: > >

Re: Spark Streaming Threading Model

2014-12-19 Thread Silvio Fiorito
Batches will wait for the previous batch to finish. The monitoring console will show you the backlog of waiting batches. From: Asim Jalis mailto:asimja...@gmail.com>> Date: Friday, December 19, 2014 at 1:16 PM To: user mailto:user@spark.apache.org>> Subject: Spark Streaming Threading Model Q: In