It is as Jagat said. The Masters do not need to know about one another, as
ZooKeeper manages their implicit communication. As for Workers (and
applications, such as spark-shell), once a Worker is registered with
*some *Master,
its metadata is stored in ZooKeeper such that if another Master is elect
ah.
standalone HA master was added in 0.9.0. Same logic, but Spark-native.
On Wed, Apr 9, 2014 at 3:31 PM, Pradeep Ch wrote:
> Thanks Dmitriy. But I want multi master support when running spark
> standalone. Also I want to know if this multi master thing works if I use
> spark-shell.
>
>
> On W
Thanks Dmitriy. But I want multi master support when running spark
standalone. Also I want to know if this multi master thing works if I use
spark-shell.
On Wed, Apr 9, 2014 at 3:26 PM, Dmitriy Lyubimov wrote:
> The only way i know to do this is to use mesos with zookeepers. you
> specify zooke
Hello Pradeep,
Quoting from
https://spark.apache.org/docs/0.9.0/spark-standalone.html
In order to schedule new applications or add Workers to the cluster, they
need to know the IP address of the current leader. This can be accomplished
by simply passing in a list of Masters where you used to pas
The only way i know to do this is to use mesos with zookeepers. you specify
zookeeper url as spark url that contains multiple zookeeper hosts. Multiple
mesos masters are then elected thru zookeeper leader election until current
leader dies; at which point mesos will elect another master (if still
l
