[
https://issues.apache.org/jira/browse/SPARK-58924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-58924:
----------------------------------
Summary: Graceful Application Hold and Resume Framework (was: Graceful
Application Suspend and Resume Framework)
> Graceful Application Hold and Resume Framework
> ----------------------------------------------
>
> Key: SPARK-58924
> URL: https://issues.apache.org/jira/browse/SPARK-58924
> Project: Spark
> Issue Type: Umbrella
> Components: Spark Core
> Affects Versions: 4.4.0
> Reporter: Dongjoon Hyun
> Assignee: Dongjoon Hyun
> Priority: Critical
> Fix For: 4.4.0
>
> Attachments: Diagram.png
>
>
> This issue aims to allow a running application to temporarily give back its
> resources without being killed. Executors are drained gracefully with no loss
> of in-progress task work or shuffle output, and the application can be
> resumed later without recomputing the shuffle output already written. Cached
> blocks are not preserved and are recomputed after resuming.
> This takes effect only when *spark.decommission.enabled* is true, the shuffle
> data is kept outside the executors (through either
> *spark.shuffle.service.enabled* or a
> ShuffleDataIO with reliable storage), and the cluster manager can hold
> executors: Standalone, YARN, and Kubernetes (with the default "direct"
> allocator).
> !Diagram.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
