Andrew Ash created SPARK-17874:
----------------------------------
Summary: Enabling SSL on HistoryServer should only open one port
not two
Key: SPARK-17874
URL: https://issues.apache.org/jira/browse/SPARK-17874
Project: Spark
Issue Type: Improvement
Components: Web UI
Affects Versions: 2.0.1
Reporter: Andrew Ash
When turning on SSL on the HistoryServer with
{{spark.ssl.historyServer.enabled=true}} this opens up a second port, at the
[hardcoded|https://github.com/apache/spark/blob/v2.0.1/core/src/main/scala/org/apache/spark/ui/JettyUtils.scala#L262]
result of calculating {{spark.history.ui.port + 400}}, and sets up a redirect
from the original (http) port to the new (https) port.
{noformat}
$ netstat -nlp | grep 23714
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp 0 0 :::18080 :::*
LISTEN 23714/java
tcp 0 0 :::18480 :::*
LISTEN 23714/java
{noformat}
By enabling {{spark.ssl.historyServer.enabled}} I would have expected the one
open port to change protocol from http to https, not to have 1) additional
ports open 2) the http port remain open 3) the additional port at a value I
didn't specify.
To fix this could take one of two approaches:
Approach 1:
- one port always, which is configured with {{spark.history.ui.port}}
- the protocol on that port is http by default
- or if {{spark.ssl.historyServer.enabled=true}} then it's https
Approach 2:
- add a new configuration item {{spark.history.ui.sslPort}} which configures
the second port that starts up
In approach 1 we probably need a way to specify to Spark jobs whether the
history server has ssl or not, based on SPARK-16988
That makes me think we should go with approach 2.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
