[
https://issues.apache.org/jira/browse/SPARK-17771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Ihbe updated SPARK-17771:
------------------------------
Description:
Based on conversation from this thread:
http://apache-spark-developers-list.1001551.n3.nabble.com/Running-Spark-master-slave-instances-in-non-Daemon-mode-td19172.html
Some scheduler solutions like Nomad have a simple fork-exec execution model,
and the daemonization causes the scheduler to lose track of the spark process.
I'm proposing adding a SPARK_NO_DAEMONIZE environment variable that will
trigger a switch in ./sbin/spark-daemon.sh to run the process in the
foreground.
This opens a question about whether or not to rename the bash script, but I
think that's a potentially breaking change that we should avoid at this point.
> Allow start-master/slave scripts to start in the foreground
> -----------------------------------------------------------
>
> Key: SPARK-17771
> URL: https://issues.apache.org/jira/browse/SPARK-17771
> Project: Spark
> Issue Type: Improvement
> Affects Versions: 2.0.1
> Reporter: Mike Ihbe
> Priority: Minor
>
> Based on conversation from this thread:
> http://apache-spark-developers-list.1001551.n3.nabble.com/Running-Spark-master-slave-instances-in-non-Daemon-mode-td19172.html
> Some scheduler solutions like Nomad have a simple fork-exec execution model,
> and the daemonization causes the scheduler to lose track of the spark
> process. I'm proposing adding a SPARK_NO_DAEMONIZE environment variable that
> will trigger a switch in ./sbin/spark-daemon.sh to run the process in the
> foreground.
> This opens a question about whether or not to rename the bash script, but I
> think that's a potentially breaking change that we should avoid at this point.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
