Re: Receiver Fault Tolerance

2015-05-06 Thread James King
Many thanks all, your responses have been very helpful. Cheers On Wed, May 6, 2015 at 2:14 PM, ayan guha wrote: > > https://spark.apache.org/docs/latest/streaming-programming-guide.html#fault-tolerance-semantics > > > On Wed, May 6, 2015 at 10:09 PM, James King wrote: > >> In the O'reilly book

Re: Receiver Fault Tolerance

2015-05-06 Thread ayan guha
https://spark.apache.org/docs/latest/streaming-programming-guide.html#fault-tolerance-semantics On Wed, May 6, 2015 at 10:09 PM, James King wrote: > In the O'reilly book Learning Spark Chapter 10 section 24/7 Operation > > It talks about 'Receiver Fault Tolerance' > > I'm unsure of what a Recei

Re: Receiver Fault Tolerance

2015-05-06 Thread Tathagata Das
Incorrect. The receiver runs in an executor just like a any other tasks. In the cluster mode, the driver runs in a worker, however it launches executors in OTHER workers in the cluster. Its those executors running in other workers that run tasks, and also the receivers. On Wed, May 6, 2015 at 5:09

RE: Receiver Fault Tolerance

2015-05-06 Thread Evo Eftimov
This is about Kafka Receiver IF you are using Spark Streaming Ps: that book is now behind the curve in a quite a few areas since the release of 1.3.1 – read the documentation and forums From: James King [mailto:jakwebin...@gmail.com] Sent: Wednesday, May 6, 2015 1:09 PM To: user Subjec