Hi guys,
That's great to hear that this is available in Spark 1.3! .. I will play
around with this feature and let you know the results for integrating
Hazelcast. Also, may I know the tentative release date for Spark 1.3? ..
Cheers,
Anjana.
On Mon, Feb 2, 2015 at 3:07 AM, Aaron Davidson wrote:
For the specific question of supplementing Standalone Mode with a custom
leader election protocol, this was actually already committed in master and
will be available in Spark 1.3:
https://github.com/apache/spark/pull/771/files
You can specify spark.deploy.recoveryMode = "CUSTOM"
and spark.deploy
+1
On a related note, there is a lot of interest in Hadoop and Spark from
the HPC community who often run slurm, pbs, and sge to control jobs (as
opposed to Yarn and Mesos). Currently, there are several projects that
launch Yarn clusters (or MR1 clusters) inside PBS Jobs [1] but this is
not th
