Hi all,

I'm trying to get the jobserver working with Spark 1.0.1. I've got it
building, tests passing and it connects to my Spark master (e.g.
spark://hadoop-001:7077).

I can also pre-create contexts. These show up in the Spark master console
i.e. on hadoop-001:8080

The problem is that after I create a context, it seems to get killed and the
job server is pushed off by the master. I see in the job server logs before
it dies;

[2014-07-31 00:31:40,166] ERROR parkDeploySchedulerBackend [] [] -
Application has been killed. Reason: Master removed our application: FAILED
[2014-07-31 00:31:40,167] ERROR cheduler.TaskSchedulerImpl [] [] - Exiting
due to error from cluster scheduler: Master removed our application: FAILED


*When the job-server starts it all looks good *

[2014-07-31 00:46:02,994] INFO  spark.jobserver.WebApi [] [] - Starting
browser web service...
[2014-07-31 00:46:03,556] INFO  ay.can.server.HttpListener []
[akka://JobServer/user/IO-HTTP/listener-0] - Bound to /0.0.0.0:8090
[2014-07-31 00:46:46,623] INFO  ocalContextSupervisorActor []
[akka://JobServer/user/context-supervisor] - Creating a SparkContext named
test-context
[2014-07-31 00:46:46,626] INFO  .jobserver.JobManagerActor []
[akka://JobServer/user/context-supervisor/test-context] - Starting actor
spark.jobserver.JobManagerActor
[2014-07-31 00:46:46,631] INFO  k.jobserver.JobStatusActor []
[akka://JobServer/user/context-supervisor/test-context/status-actor] -
Starting actor spark.jobserver.JobStatusActor
[2014-07-31 00:46:46,632] INFO  k.jobserver.JobResultActor []
[akka://JobServer/user/context-supervisor/test-context/result-actor] -
Starting actor spark.jobserver.JobResultActor
[2014-07-31 00:46:46,674] INFO  ache.spark.SecurityManager []
[akka://JobServer/user/context-supervisor/test-context] - Changing view acls
to: nathan
[2014-07-31 00:46:46,675] INFO  ache.spark.SecurityManager []
[akka://JobServer/user/context-supervisor/test-context] - SecurityManager:
authentication disabled; ui acls disabled; users with view permissions:
Set(nathan)
[2014-07-31 00:46:46,755] INFO  ka.event.slf4j.Slf4jLogger []
[akka://JobServer/user/context-supervisor/test-context] - Slf4jLogger
started
[2014-07-31 00:46:46,784] INFO  Remoting [] [Remoting] - Starting remoting
[2014-07-31 00:46:46,910] INFO  Remoting [] [Remoting] - Remoting started;
listening on addresses :[akka.tcp://spark@localhost:38992]
[2014-07-31 00:46:46,912] INFO  Remoting [] [Remoting] - Remoting now
listens on addresses: [akka.tcp://spark@localhost:38992]
[2014-07-31 00:46:46,932] INFO  org.apache.spark.SparkEnv []
[akka://JobServer/user/context-supervisor/test-context] - Registering
MapOutputTracker
[2014-07-31 00:46:46,934] INFO  org.apache.spark.SparkEnv []
[akka://JobServer/user/context-supervisor/test-context] - Registering
BlockManagerMaster
[2014-07-31 00:46:46,953] INFO  k.storage.DiskBlockManager []
[akka://JobServer/user/context-supervisor/test-context] - Created local
directory at /tmp/spark-local-20140731004646-5417
[2014-07-31 00:46:46,956] INFO  .spark.storage.MemoryStore []
[akka://JobServer/user/context-supervisor/test-context] - MemoryStore
started with capacity 15.9 GB.
[2014-07-31 00:46:46,973] INFO  .network.ConnectionManager []
[akka://JobServer/user/context-supervisor/test-context] - Bound socket to
port 36182 with id = ConnectionManagerId(localhost,36182)
[2014-07-31 00:46:46,977] INFO  storage.BlockManagerMaster []
[akka://JobServer/user/context-supervisor/test-context] - Trying to register
BlockManager
[2014-07-31 00:46:46,979] INFO  k.storage.BlockManagerInfo [] [] -
Registering block manager localhost:36182 with 15.9 GB RAM
[2014-07-31 00:46:46,980] INFO  storage.BlockManagerMaster []
[akka://JobServer/user/context-supervisor/test-context] - Registered
BlockManager
[2014-07-31 00:46:46,994] INFO  rg.apache.spark.HttpServer []
[akka://JobServer/user/context-supervisor/test-context] - Starting HTTP
Server
[2014-07-31 00:46:47,068] INFO  clipse.jetty.server.Server []
[akka://JobServer/user/context-supervisor/test-context] -
jetty-8.1.14.v20131031
[2014-07-31 00:46:47,080] INFO  y.server.AbstractConnector []
[akka://JobServer/user/context-supervisor/test-context] - Started
[email protected]:54907
[2014-07-31 00:46:47,081] INFO  rk.broadcast.HttpBroadcast []
[akka://JobServer/user/context-supervisor/test-context] - Broadcast server
started at http://localhost:54907
[2014-07-31 00:46:47,086] INFO  pache.spark.HttpFileServer []
[akka://JobServer/user/context-supervisor/test-context] - HTTP File server
directory is /tmp/spark-15f9ba3c-79f9-497d-b4eb-a79cc9b0258c
[2014-07-31 00:46:47,086] INFO  rg.apache.spark.HttpServer []
[akka://JobServer/user/context-supervisor/test-context] - Starting HTTP
Server
[2014-07-31 00:46:47,087] INFO  clipse.jetty.server.Server []
[akka://JobServer/user/context-supervisor/test-context] -
jetty-8.1.14.v20131031
[2014-07-31 00:46:47,088] INFO  y.server.AbstractConnector []
[akka://JobServer/user/context-supervisor/test-context] - Started
[email protected]:36085
[2014-07-31 00:46:47,311] INFO  clipse.jetty.server.Server []
[akka://JobServer/user/context-supervisor/test-context] -
jetty-8.1.14.v20131031
[2014-07-31 00:46:47,320] INFO  y.server.AbstractConnector []
[akka://JobServer/user/context-supervisor/test-context] - Started
[email protected]:42107
[2014-07-31 00:46:47,322] INFO  rg.apache.spark.ui.SparkUI []
[akka://JobServer/user/context-supervisor/test-context] - Started SparkUI at
http://localhost:42107
[2014-07-31 00:46:47,554] INFO  ient.AppClient$ClientActor [] [] -
Connecting to master spark://hadoop-001:7077...
[2014-07-31 00:46:47,579] INFO  .jobserver.RddManagerActor []
[akka://JobServer/user/context-supervisor/test-context/rdd-manager-actor] -
Starting actor spark.jobserver.RddManagerActor
[2014-07-31 00:46:47,582] INFO  ocalContextSupervisorActor [] [] -
SparkContext test-context initialized
[2014-07-31 00:46:47,728] INFO  parkDeploySchedulerBackend [] [] - Connected
to Spark cluster with app ID app-20140731004647-0096
[2014-07-31 00:46:47,729] INFO  ient.AppClient$ClientActor [] [] - Executor
added: app-20140731004647-0096/0 on worker-20140730213646-hadoop-006-41606
(hadoop-006:41606) with 1 cores
[2014-07-31 00:46:47,730] INFO  parkDeploySchedulerBackend [] [] - Granted
executor ID app-20140731004647-0096/0 on hostPort hadoop-006:41606 with 1
cores, 512.0 MB RAM
[2014-07-31 00:46:47,730] INFO  ient.AppClient$ClientActor [] [] - Executor
added: app-20140731004647-0096/1 on worker-20140730235527-hadoop-005-55593
(hadoop-005:55593) with 1 cores
[2014-07-31 00:46:47,731] INFO  parkDeploySchedulerBackend [] [] - Granted
executor ID app-20140731004647-0096/1 on hostPort hadoop-005:55593 with 1
cores, 512.0 MB RAM
[2014-07-31 00:46:47,731] INFO  ient.AppClient$ClientActor [] [] - Executor
added: app-20140731004647-0096/2 on worker-20140730213651-hadoop-003-56132
(hadoop-003:56132) with 1 cores
[2014-07-31 00:46:47,731] INFO  parkDeploySchedulerBackend [] [] - Granted
executor ID app-20140731004647-0096/2 on hostPort hadoop-003:56132 with 1
cores, 512.0 MB RAM
[2014-07-31 00:46:47,731] INFO  ient.AppClient$ClientActor [] [] - Executor
added: app-20140731004647-0096/3 on worker-20140730213607-hadoop-004-40125
(hadoop-004:40125) with 1 cores
[2014-07-31 00:46:47,731] INFO  parkDeploySchedulerBackend [] [] - Granted
executor ID app-20140731004647-0096/3 on hostPort hadoop-004:40125 with 1
cores, 512.0 MB RAM
[2014-07-31 00:46:47,738] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/1 is now RUNNING
[2014-07-31 00:46:47,738] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/0 is now RUNNING
[2014-07-31 00:46:47,740] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/3 is now RUNNING
[2014-07-31 00:46:47,741] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/2 is now RUNNING


*But then after a while I see;*

[2014-07-31 00:47:20,613] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/2 is now EXITED (Command exited with code
1)
[2014-07-31 00:47:20,615] INFO  parkDeploySchedulerBackend [] [] - Executor
app-20140731004647-0096/2 removed: Command exited with code 1
[2014-07-31 00:47:20,616] INFO  ient.AppClient$ClientActor [] [] - Executor
added: app-20140731004647-0096/4 on worker-20140730213651-hadoop-003-56132
(hadoop-003:56132) with 1 cores
[2014-07-31 00:47:20,616] INFO  parkDeploySchedulerBackend [] [] - Granted
executor ID app-20140731004647-0096/4 on hostPort hadoop-003:56132 with 1
cores, 512.0 MB RAM
[2014-07-31 00:47:20,617] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/0 is now EXITED (Command exited with code
1)
[2014-07-31 00:47:20,617] INFO  parkDeploySchedulerBackend [] [] - Executor
app-20140731004647-0096/0 removed: Command exited with code 1
[2014-07-31 00:47:20,617] INFO  ient.AppClient$ClientActor [] [] - Executor
added: app-20140731004647-0096/5 on worker-20140730213646-hadoop-006-41606
(hadoop-006:41606) with 1 cores
[2014-07-31 00:47:20,618] INFO  parkDeploySchedulerBackend [] [] - Granted
executor ID app-20140731004647-0096/5 on hostPort hadoop-006:41606 with 1
cores, 512.0 MB RAM
[2014-07-31 00:47:20,618] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/5 is now RUNNING
[2014-07-31 00:47:20,620] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/4 is now RUNNING
[2014-07-31 00:47:20,630] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/3 is now EXITED (Command exited with code
1)
[2014-07-31 00:47:20,630] INFO  parkDeploySchedulerBackend [] [] - Executor
app-20140731004647-0096/3 removed: Command exited with code 1
[2014-07-31 00:47:20,630] INFO  ient.AppClient$ClientActor [] [] - Executor
added: app-20140731004647-0096/6 on worker-20140730213607-hadoop-004-40125
(hadoop-004:40125) with 1 cores
[2014-07-31 00:47:20,630] INFO  parkDeploySchedulerBackend [] [] - Granted
executor ID app-20140731004647-0096/6 on hostPort hadoop-004:40125 with 1
cores, 512.0 MB RAM
[2014-07-31 00:47:20,633] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/1 is now EXITED (Command exited with code
1)
[2014-07-31 00:47:20,633] INFO  parkDeploySchedulerBackend [] [] - Executor
app-20140731004647-0096/1 removed: Command exited with code 1
[2014-07-31 00:47:20,633] INFO  ient.AppClient$ClientActor [] [] - Executor
added: app-20140731004647-0096/7 on worker-20140730235527-hadoop-005-55593
(hadoop-005:55593) with 1 cores
[2014-07-31 00:47:20,634] INFO  parkDeploySchedulerBackend [] [] - Granted
executor ID app-20140731004647-0096/7 on hostPort hadoop-005:55593 with 1
cores, 512.0 MB RAM
[2014-07-31 00:47:20,636] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/6 is now RUNNING
[2014-07-31 00:47:20,640] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/7 is now RUNNING


*And again;*

[2014-07-31 00:47:53,514] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/4 is now EXITED (Command exited with code
1)
[2014-07-31 00:47:53,514] INFO  parkDeploySchedulerBackend [] [] - Executor
app-20140731004647-0096/4 removed: Command exited with code 1
[2014-07-31 00:47:53,515] INFO  ient.AppClient$ClientActor [] [] - Executor
added: app-20140731004647-0096/8 on worker-20140730213651-hadoop-003-56132
(hadoop-003:56132) with 1 cores
[2014-07-31 00:47:53,515] INFO  parkDeploySchedulerBackend [] [] - Granted
executor ID app-20140731004647-0096/8 on hostPort hadoop-003:56132 with 1
cores, 512.0 MB RAM
[2014-07-31 00:47:53,519] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/8 is now RUNNING
[2014-07-31 00:47:53,528] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/5 is now EXITED (Command exited with code
1)
[2014-07-31 00:47:53,528] INFO  parkDeploySchedulerBackend [] [] - Executor
app-20140731004647-0096/5 removed: Command exited with code 1
[2014-07-31 00:47:53,529] INFO  ient.AppClient$ClientActor [] [] - Executor
added: app-20140731004647-0096/9 on worker-20140730213646-hadoop-006-41606
(hadoop-006:41606) with 1 cores
[2014-07-31 00:47:53,529] INFO  parkDeploySchedulerBackend [] [] - Granted
executor ID app-20140731004647-0096/9 on hostPort hadoop-006:41606 with 1
cores, 512.0 MB RAM
[2014-07-31 00:47:53,534] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/9 is now RUNNING
[2014-07-31 00:47:53,563] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/6 is now EXITED (Command exited with code
1)
[2014-07-31 00:47:53,563] INFO  parkDeploySchedulerBackend [] [] - Executor
app-20140731004647-0096/6 removed: Command exited with code 1
[2014-07-31 00:47:53,563] INFO  ient.AppClient$ClientActor [] [] - Executor
added: app-20140731004647-0096/10 on worker-20140730213607-hadoop-004-40125
(hadoop-004:40125) with 1 cores
[2014-07-31 00:47:53,563] INFO  parkDeploySchedulerBackend [] [] - Granted
executor ID app-20140731004647-0096/10 on hostPort hadoop-004:40125 with 1
cores, 512.0 MB RAM
[2014-07-31 00:47:53,568] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/10 is now RUNNING
[2014-07-31 00:47:53,590] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/7 is now EXITED (Command exited with code
1)
[2014-07-31 00:47:53,590] INFO  parkDeploySchedulerBackend [] [] - Executor
app-20140731004647-0096/7 removed: Command exited with code 1
[2014-07-31 00:47:53,590] INFO  ient.AppClient$ClientActor [] [] - Executor
added: app-20140731004647-0096/11 on worker-20140730235527-hadoop-005-55593
(hadoop-005:55593) with 1 cores
[2014-07-31 00:47:53,590] INFO  parkDeploySchedulerBackend [] [] - Granted
executor ID app-20140731004647-0096/11 on hostPort hadoop-005:55593 with 1
cores, 512.0 MB RAM
[2014-07-31 00:47:53,595] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/11 is now RUNNING


*Then it dies;*

[2014-07-31 00:48:26,446] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/8 is now EXITED (Command exited with code
1)
[2014-07-31 00:48:26,446] INFO  parkDeploySchedulerBackend [] [] - Executor
app-20140731004647-0096/8 removed: Command exited with code 1
[2014-07-31 00:48:26,447] INFO  ient.AppClient$ClientActor [] [] - Executor
added: app-20140731004647-0096/12 on worker-20140730213651-hadoop-003-56132
(hadoop-003:56132) with 1 cores
[2014-07-31 00:48:26,447] INFO  parkDeploySchedulerBackend [] [] - Granted
executor ID app-20140731004647-0096/12 on hostPort hadoop-003:56132 with 1
cores, 512.0 MB RAM
[2014-07-31 00:48:26,451] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/12 is now RUNNING
[2014-07-31 00:48:26,453] INFO  ient.AppClient$ClientActor [] [] - Executor
updated: app-20140731004647-0096/10 is now EXITED (Command exited with code
1)
[2014-07-31 00:48:26,453] INFO  parkDeploySchedulerBackend [] [] - Executor
app-20140731004647-0096/10 removed: Command exited with code 1
[2014-07-31 00:48:26,454] ERROR parkDeploySchedulerBackend [] [] -
Application has been killed. Reason: Master removed our application: FAILED
[2014-07-31 00:48:26,455] ERROR cheduler.TaskSchedulerImpl [] [] - Exiting
due to error from cluster scheduler: Master removed our application: FAILED


*The Spark master logs which don't give away a lot of info.*

14/07/31 00:46:47 INFO master.Master: Registering app test-context
14/07/31 00:46:47 INFO master.Master: Registered app test-context with ID
app-20140731004647-0096
14/07/31 00:46:47 INFO master.Master: Launching executor
app-20140731004647-0096/0 on worker worker-20140730213646-hadoop-006-41606
14/07/31 00:46:47 INFO master.Master: Launching executor
app-20140731004647-0096/1 on worker worker-20140730235527-hadoop-005-55593
14/07/31 00:46:47 INFO master.Master: Launching executor
app-20140731004647-0096/2 on worker worker-20140730213651-hadoop-003-56132
14/07/31 00:46:47 INFO master.Master: Launching executor
app-20140731004647-0096/3 on worker worker-20140730213607-hadoop-004-40125
14/07/31 00:47:20 INFO master.Master: Removing executor
app-20140731004647-0096/2 because it is EXITED
14/07/31 00:47:20 INFO master.Master: Launching executor
app-20140731004647-0096/4 on worker worker-20140730213651-hadoop-003-56132
14/07/31 00:47:20 INFO master.Master: Removing executor
app-20140731004647-0096/0 because it is EXITED
14/07/31 00:47:20 INFO master.Master: Launching executor
app-20140731004647-0096/5 on worker worker-20140730213646-hadoop-006-41606
14/07/31 00:47:20 INFO master.Master: Removing executor
app-20140731004647-0096/3 because it is EXITED
14/07/31 00:47:20 INFO master.Master: Launching executor
app-20140731004647-0096/6 on worker worker-20140730213607-hadoop-004-40125
14/07/31 00:47:20 INFO master.Master: Removing executor
app-20140731004647-0096/1 because it is EXITED
14/07/31 00:47:20 INFO master.Master: Launching executor
app-20140731004647-0096/7 on worker worker-20140730235527-hadoop-005-55593
14/07/31 00:47:41 ERROR remote.EndpointWriter: AssociationError
[akka.tcp://sparkMaster@hadoop-001:7077] ->
[akka.tcp://[email protected]:58184]: Error [Association failed with
[akka.tcp://[email protected]:58184]] [
akka.remote.EndpointAssociationException: Association failed with
[akka.tcp://[email protected]:58184]
Caused by:
akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2:
connection timed out: /10.10.0.69:58184
]
14/07/31 00:47:41 INFO master.Master: akka.tcp://[email protected]:58184 got
disassociated, removing it.
14/07/31 00:47:53 INFO master.Master: Removing executor
app-20140731004647-0096/4 because it is EXITED
14/07/31 00:47:53 INFO master.Master: Launching executor
app-20140731004647-0096/8 on worker worker-20140730213651-hadoop-003-56132
14/07/31 00:47:53 INFO master.Master: Removing executor
app-20140731004647-0096/5 because it is EXITED
14/07/31 00:47:53 INFO master.Master: Launching executor
app-20140731004647-0096/9 on worker worker-20140730213646-hadoop-006-41606
14/07/31 00:47:53 INFO master.Master: Removing executor
app-20140731004647-0096/6 because it is EXITED
14/07/31 00:47:53 INFO master.Master: Launching executor
app-20140731004647-0096/10 on worker worker-20140730213607-hadoop-004-40125
14/07/31 00:47:53 INFO master.Master: Removing executor
app-20140731004647-0096/7 because it is EXITED
14/07/31 00:47:53 INFO master.Master: Launching executor
app-20140731004647-0096/11 on worker worker-20140730235527-hadoop-005-55593
14/07/31 00:48:26 INFO master.Master: Removing executor
app-20140731004647-0096/8 because it is EXITED
14/07/31 00:48:26 INFO master.Master: Launching executor
app-20140731004647-0096/12 on worker worker-20140730213651-hadoop-003-56132
14/07/31 00:48:26 INFO master.Master: Removing executor
app-20140731004647-0096/10 because it is EXITED
14/07/31 00:48:26 ERROR master.Master: Application test-context with ID
app-20140731004647-0096 failed 10 times, removing it
14/07/31 00:48:26 INFO master.Master: Removing app app-20140731004647-0096
14/07/31 00:48:26 WARN master.Master: Got status update for unknown executor
app-20140731004647-0096/12
14/07/31 00:48:26 WARN master.Master: Got status update for unknown executor
app-20140731004647-0096/9
14/07/31 00:48:26 WARN master.Master: Got status update for unknown executor
app-20140731004647-0096/11
14/07/31 00:48:26 INFO master.Master: akka.tcp://spark@localhost:38992 got
disassociated, removing it.
14/07/31 00:48:26 INFO master.Master: akka.tcp://spark@localhost:38992 got
disassociated, removing it.
14/07/31 00:48:26 INFO actor.LocalActorRef: Message
[akka.remote.transport.ActorTransportAdapter$DisassociateUnderlying] from
Actor[akka://sparkMaster/deadLetters] to
Actor[akka://sparkMaster/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2FsparkMaster%40192.168.80.20%3A42980-2267#1228619927]
was not delivered. [436] dead letters encountered. This logging can be
turned off or adjusted with configuration settings 'akka.log-dead-letters'
and 'akka.log-dead-letters-during-shutdown'.
14/07/31 00:48:26 ERROR remote.EndpointWriter: AssociationError
[akka.tcp://sparkMaster@hadoop-001:7077] ->
[akka.tcp://spark@localhost:38992]: Error [Association failed with
[akka.tcp://spark@localhost:38992]] [
akka.remote.EndpointAssociationException: Association failed with
[akka.tcp://spark@localhost:38992]
Caused by:
akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2:
Connection refused: localhost/127.0.0.1:38992
]
14/07/31 00:48:26 INFO master.Master: akka.tcp://spark@localhost:38992 got
disassociated, removing it.
14/07/31 00:48:26 ERROR remote.EndpointWriter: AssociationError
[akka.tcp://sparkMaster@hadoop-001:7077] ->
[akka.tcp://spark@localhost:38992]: Error [Association failed with
[akka.tcp://spark@localhost:38992]] [
akka.remote.EndpointAssociationException: Association failed with
[akka.tcp://spark@localhost:38992]
Caused by:
akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2:
Connection refused: localhost/127.0.0.1:38992
]
14/07/31 00:48:26 INFO master.Master: akka.tcp://spark@localhost:38992 got
disassociated, removing it.
14/07/31 00:48:26 ERROR remote.EndpointWriter: AssociationError
[akka.tcp://sparkMaster@hadoop-001:7077] ->
[akka.tcp://spark@localhost:38992]: Error [Association failed with
[akka.tcp://spark@localhost:38992]] [
akka.remote.EndpointAssociationException: Association failed with
[akka.tcp://spark@localhost:38992]
Caused by:
akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2:
Connection refused: localhost/127.0.0.1:38992
]
14/07/31 00:48:26 INFO master.Master: akka.tcp://spark@localhost:38992 got
disassociated, removing it.
14/07/31 00:49:21 ERROR remote.EndpointWriter: AssociationError
[akka.tcp://sparkMaster@hadoop-001:7077] ->
[akka.tcp://[email protected]:58184]: Error [Association failed with
[akka.tcp://[email protected]:58184]] [
akka.remote.EndpointAssociationException: Association failed with
[akka.tcp://[email protected]:58184]
Caused by:
akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2:
connection timed out: /10.10.0.69:58184
]
14/07/31 00:49:21 INFO master.Master: akka.tcp://[email protected]:58184 got
disassociated, removing it.
14/07/31 00:51:01 ERROR remote.EndpointWriter: AssociationError
[akka.tcp://sparkMaster@hadoop-001:7077] ->
[akka.tcp://[email protected]:58184]: Error [Association failed with
[akka.tcp://[email protected]:58184]] [
akka.remote.EndpointAssociationException: Association failed with
[akka.tcp://[email protected]:58184]
Caused by:
akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2:
connection timed out: /10.10.0.69:58184
]
14/07/31 00:51:01 INFO master.Master: akka.tcp://[email protected]:58184 got
disassociated, removing it.
14/07/31 00:52:41 ERROR remote.EndpointWriter: AssociationError
[akka.tcp://sparkMaster@hadoop-001:7077] ->
[akka.tcp://[email protected]:58184]: Error [Association failed with
[akka.tcp://[email protected]:58184]] [
akka.remote.EndpointAssociationException: Association failed with
[akka.tcp://[email protected]:58184]
Caused by:
akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2:
connection timed out: /10.10.0.69:58184
]
14/07/31 00:52:41 INFO master.Master: akka.tcp://[email protected]:58184 got
disassociated, removing it.


Any ideas on whats going on here? 



--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Ooyala-Job-Server-tp10936.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Reply via email to