Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/2752#issuecomment-58655305
This will cause some bug.
It seems to be related to `akka.remote.retry-gate-closed-for` configuration
the log:
```
14/10/10 20:56:03 WARN ReliableDeliverySupervisor: Association with remote
system [akka.tcp://[email protected]:20412] has failed, address
is now gated for [5000] ms. Reason is: [Disassociated].
14/10/10 20:56:03 INFO TaskSetManager: Starting task 60.0 in stage 5427.0
(TID 35496, 10dian71.domain.test, PROCESS_LOCAL, 8062 bytes)
14/10/10 20:56:03 INFO TaskSetManager: Finished task 39.0 in stage 5427.0
(TID 35480) in 539 ms on 10dian71.domain.test (63/72)
14/10/10 20:56:03 INFO TaskSetManager: Finished task 50.0 in stage 5427.0
(TID 35488) in 279 ms on 10dian72.domain.test (64/72)
14/10/10 20:56:03 ERROR YarnClientSchedulerBackend: Yarn application has
already exited with state FINISHED!
14/10/10 20:56:03 INFO SparkUI: Stopped Spark web UI at
http://10dian71.domain.test:4040
14/10/10 20:56:03 INFO DAGScheduler: Stopping DAGScheduler
14/10/10 20:56:03 INFO YarnClientSchedulerBackend: Shutting down all
executors
14/10/10 20:56:03 INFO YarnClientSchedulerBackend: Asking each executor to
shut down
14/10/10 20:56:03 INFO DAGScheduler: Job 218 failed: count at
TopicModeling.scala:475, took 2.400066 s
14/10/10 20:56:03 INFO YarnClientSchedulerBackend: Stopped
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
