Github user devaraj-kavali commented on the pull request:
https://github.com/apache/spark/pull/11133#issuecomment-187096991
I see the test/Jenkins failure is due to the PR change.
Here org.apache.spark.deploy.LogUrlsStandaloneSuite is failing because of
the below exception,
```
16/02/22 17:38:32.257 dispatcher-event-loop-5 ERROR Worker: Connection to
master failed! Waiting for master to reconnect...
16/02/22 17:38:42.441 ScalaTest-main-running-LogUrlsStandaloneSuite ERROR
SparkUI: Failed to bind SparkUI
java.net.SocketException: Unresolved address
at sun.nio.ch.Net.translateToSocketException(Net.java:157)
at sun.nio.ch.Net.translateException(Net.java:183)
at sun.nio.ch.Net.translateException(Net.java:189)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:76)
at
org.eclipse.jetty.server.nio.SelectChannelConnector.open(SelectChannelConnector.java:187)
at
org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:316)
at
org.eclipse.jetty.server.nio.SelectChannelConnector.doStart(SelectChannelConnector.java:265)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
at org.eclipse.jetty.server.Server.doStart(Server.java:293)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
at
org.apache.spark.ui.JettyUtils$.org$apache$spark$ui$JettyUtils$$connect$1(JettyUtils.scala:283)
at org.apache.spark.ui.JettyUtils$$anonfun$5.apply(JettyUtils.scala:293)
at org.apache.spark.ui.JettyUtils$$anonfun$5.apply(JettyUtils.scala:293)
at
org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:1973)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:166)
at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:1964)
at
org.apache.spark.ui.JettyUtils$.startJettyServer(JettyUtils.scala:293)
at org.apache.spark.ui.WebUI.bind(WebUI.scala:137)
at
org.apache.spark.SparkContext$$anonfun$13.apply(SparkContext.scala:458)
at
org.apache.spark.SparkContext$$anonfun$13.apply(SparkContext.scala:458)
at scala.Option.foreach(Option.scala:257)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:458)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:133)
at
org.apache.spark.deploy.LogUrlsStandaloneSuite$$anonfun$2.apply$mcV$sp(LogUrlsStandaloneSuite.scala:59)
at
org.apache.spark.deploy.LogUrlsStandaloneSuite$$anonfun$2.apply(LogUrlsStandaloneSuite.scala:55)
at
org.apache.spark.deploy.LogUrlsStandaloneSuite$$anonfun$2.apply(LogUrlsStandaloneSuite.scala:55)
at
org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
```
In LogUrlsStandaloneSuite.scala, SPARK_PUBLIC_DNS is getting set as
"public_dns" and same is trying to use while starting the jetty server in
WebUI.scala and it is failing to resolve the "public_dns".
```javascript
test("verify that log urls reflect SPARK_PUBLIC_DNS (SPARK-6175)") {
val SPARK_PUBLIC_DNS = "public_dns"
val conf = new SparkConfWithEnv(Map("SPARK_PUBLIC_DNS" ->
SPARK_PUBLIC_DNS)).set(
"spark.extraListeners", classOf[SaveExecutorInfo].getName)
sc = new SparkContext("local-cluster[2,1,1024]", "test", conf)
```
```javascript
protected val publicHostName =
Option(conf.getenv("SPARK_PUBLIC_DNS")).getOrElse(localHostName)
........
def bind() {
assert(!serverInfo.isDefined, "Attempted to bind %s more than
once!".format(className))
try {
serverInfo = Some(startJettyServer(publicHostName, port, sslOptions,
handlers, conf, name))
logInfo("Started %s at http://%s:%d".format(className,
publicHostName, boundPort))
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
