I guess this is known issue and being tracked by SPARK-15544
[1] SPARK-23530 [2] (seems duplicated)
I guess that's the simplest implementation of H/A (since we don't bother
with current state in master) when background process like supervisord
restarts the process when process is no longer running
As I understand, Apache Spark Master can be run in high availability mode
using Zookeeper. That is, multiple Spark masters can run in Leader/Follower
mode and these modes are registered with Zookeeper.
In our scenario Zookeeper is expiring the Spark Master's session which is
acting as Leader. So t
