Re: driver fail-over in Spark streaming 1.2.0

2015-02-12 Thread Patrick Wendell
It will create and connect to new executors. The executors are mostly stateless, so the program can resume with new executors. On Wed, Feb 11, 2015 at 11:24 PM, lin wrote: > Hi, all > > In Spark Streaming 1.2.0, when the driver fails and a new driver starts > with the most updated check-pointed d

driver fail-over in Spark streaming 1.2.0

2015-02-11 Thread lin
Hi, all In Spark Streaming 1.2.0, when the driver fails and a new driver starts with the most updated check-pointed data, will the former Executors connects to the new driver, or will the new driver starts out its own set of new Executors? In which piece of codes is that done? Any reply will be a