0193
China
Tathagata Das
2014/12/11 20:20
To
Jun Feng Liu/China/IBM@IBMCN,
cc
Sandy Ryza , "dev@spark.apache.org"
, Reynold Xin
Subject
Re: HA support for Spark
Spark Streaming essentially does this by saving the DAG of DStreams, which
can deterministically regener
01:36:35---Sandy Ryza
>
>
>*Sandy Ryza >*
>
>2014-12-11 01:34
>
>
> To
>
>
>Jun Feng Liu/China/IBM@IBMCN,
>
>
> cc
>
>
>Reynold Xin , "dev@spark.apache.org" <
>dev@spark.apache.org>
>
>
> Subject
&
e.org"
Subject
Re: HA support
h contact information] *Phone:
> *86-10-82452683
>
> * E-mail:* *liuj...@cn.ibm.com*
> [image: IBM]
>
> BLD 28,ZGC Software Park
> No.8 Rd.Dong Bei Wang West, Dist.Haidian Beijing 100193
> China
>
>
>
>
>
> *Reynold Xin >*
>
> 2014/12/10 16:30
> To
.com
BLD 28,ZGC Software Park
No.8 Rd.Dong Bei Wang West, Dist.Haidian Beijing 100193
China
Reynold Xin
2014/12/10 16:30
To
Jun Feng Liu/China/IBM@IBMCN,
cc
"dev@spark.apache.org"
Subject
Re: HA support for Spark
This would be plausible for specific purposes such as S
This would be plausible for specific purposes such as Spark streaming or
Spark SQL, but I don't think it is doable for general Spark driver since it
is just a normal JVM process with arbitrary program state.
On Wed, Dec 10, 2014 at 12:25 AM, Jun Feng Liu wrote:
> Do we have any high availability
Do we have any high availability support in Spark driver level? For
example, if we want spark drive can move to another node continue
execution when failure happen. I can see the RDD checkpoint can help to
serialization the status of RDD. I can image to load the check point from
another node wh
