[
https://issues.apache.org/jira/browse/SPARK-16455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
YangyangLiu updated SPARK-16455:
--------------------------------
Description: In our case, we are implementing a new mechanism which will
let driver survive when cluster is temporarily down and restarting. So when the
service provided by cluster is not available, scheduler should stop scheduling
new tasks. I added a hook inside CoarseGrainedSchedulerBackend class, in order
to avoid new task scheduling when it's necessary. (was: In our case, we are
implementing restartable resource manager mechanism which will let master
executor survive when resource manager service is temporarily down. So when
resource manager service is not available, scheduler should stop scheduling new
tasks. I added a hook inside CoarseGrainedSchedulerBackend class, in order to
avoid new task scheduling when it's necessary.)
Summary: Add a new hook in CoarseGrainedSchedulerBackend in order to
stop scheduling new tasks when cluster is restarting (was: Add a new hook in
makeOffers in CoarseGrainedSchedulerBackend)
> Add a new hook in CoarseGrainedSchedulerBackend in order to stop scheduling
> new tasks when cluster is restarting
> ----------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-16455
> URL: https://issues.apache.org/jira/browse/SPARK-16455
> Project: Spark
> Issue Type: New Feature
> Components: Scheduler
> Reporter: YangyangLiu
> Priority: Minor
>
> In our case, we are implementing a new mechanism which will let driver
> survive when cluster is temporarily down and restarting. So when the service
> provided by cluster is not available, scheduler should stop scheduling new
> tasks. I added a hook inside CoarseGrainedSchedulerBackend class, in order to
> avoid new task scheduling when it's necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
