On 9 Dec 2015, at 22:01, Kelvin Chu <[email protected]<mailto:[email protected]>> wrote:
Spark on YARN can use History Server by setting the configuration spark.yarn.historyServer.address. That's the stuff in SPARK-1537 which isn' actually built in yet. But, I can't find similar config for Mesos. Is History Server supported by Spark on Mesos? Thanks. you set up your jobs to log to HDFS and run the history server to poll and replay the logs that appear there. see: http://spark.apache.org/docs/latest/monitoring.html
