[
https://issues.apache.org/jira/browse/SPARK-51480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ruiliang updated SPARK-51480:
-----------------------------
Description:
h2. [Elastic Memory
Feature|[https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCGroupsMemory.html#:~:text=Elastic%20Memory%20Feature]]
[spark set
xmx|https://spark.apache.org/docs/latest/configuration.html#:~:text=3.0.0-,spark.executor.extraJavaOptions,-(none)]
Currently, mapreduce can be specified separately
{code:java}
mapreduce.{reduce|map}.memory-mb={xx}
mapreduce.{reduce|map}.java.opts=-Xmx{xx}m{code}
[spark That should be removed here as
well|https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkConf.scala#L570-L572]
was:
h2. [Elastic Memory Feature|
[https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCGroupsMemory.html#:~:text=Elastic%20Memory%20Feature]
h2. ]
[spark set
xmx|https://spark.apache.org/docs/latest/configuration.html#:~:text=3.0.0-,spark.executor.extraJavaOptions,-(none)]
Currently, mapreduce can be specified separately
{code:java}
mapreduce.{reduce|map}.memory-mb={xx}
mapreduce.{reduce|map}.java.opts=-Xmx{xx}m{code}
[spark That should be removed here as
well|https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkConf.scala#L570-L572]
> saprk jvm xmx and container momory need to be set separately
> ------------------------------------------------------------
>
> Key: SPARK-51480
> URL: https://issues.apache.org/jira/browse/SPARK-51480
> Project: Spark
> Issue Type: Improvement
> Components: Spark Submit
> Affects Versions: 3.5.5
> Reporter: ruiliang
> Priority: Minor
>
> h2. [Elastic Memory
> Feature|[https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCGroupsMemory.html#:~:text=Elastic%20Memory%20Feature]]
>
> [spark set
> xmx|https://spark.apache.org/docs/latest/configuration.html#:~:text=3.0.0-,spark.executor.extraJavaOptions,-(none)]
>
>
> Currently, mapreduce can be specified separately
>
> {code:java}
> mapreduce.{reduce|map}.memory-mb={xx}
> mapreduce.{reduce|map}.java.opts=-Xmx{xx}m{code}
>
> [spark That should be removed here as
> well|https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkConf.scala#L570-L572]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
