what did you mean by “link” ? an HTTP URL to the spark monitoring UI? AFAIK, it is not directly supported. i typically go to both masters and check which one is active :-)
did you check if the failover actually happened in other ways (i don’t know what the znode should say)? you can try sending a spark job and if you used the right master config in your code, it should go to the new master. that will confirm that failover worked. Mohit Jaggi Founder, Data Orchard LLC www.dataorchardllc.com > On Aug 19, 2016, at 8:56 PM, Charles Nnamdi Akalugwu <cprenzb...@gmail.com> > wrote: > > I am experiencing this exact issue. Does anyone know what's going on with the > zookeeper setup? > > On Jul 5, 2016 10:34 AM, "Akmal Abbasov" <akmal.abba...@icloud.com > <mailto:akmal.abba...@icloud.com>> wrote: > > > > Hi, > > I'm trying to understand how Spark HA works. I'm using Spark 1.6.1 and > > Zookeeper 3.4.6. > > I've add the following line to $SPARK_HOME/conf/spark-env.sh > > export SPARK_DAEMON_JAVA_OPTS="-Dspark.deploy.recoveryMode=ZOOKEEPER > > -Dspark.deploy.zookeeper.url=zk1:2181,zk2:2181,zk3:2181 > > -Dspark.deploy.zookeeper.dir=/spark > > It's working so far. > > I'd like to setup a link which will always go to active master UI(I'm using > > Spark in Standalone). > > I've checked the znode /spark, and it contains > > [leader_election, master_status] > > I'm assuming that master_status znode will contain ip address of the > > current active master, is it true? Because in my case this znode isn't > > updated after failover. > > And how /spark/leader_election works, because it doesn't contain any data. > > Thank you. > > > > Regards, > > Akmal > > > > >
