Hi,

There is a cpu parameter to set spark executor on k8s
spark.kubernetes.executor.limit.cores and
spark.kubernetes.executor.request.cores
but there is no parameter to set memory request different then limits
memory (such as spark.kubernetes.executor.request.memory)
For that reason,
spark.executor.memory is assigned to  requests.memory and limits.memory
like the following

Limits:
  memory:  5734MiRequests:
  cpu:     4
  memory:  5734Mi


Is there any special reason to not have
spark.kubernetes.executor.request.memory parameter?
and can I use spark.kubernetes.executor.podTemplateFile parameter to set
smaller memory request than the memory limit in pod template file?


Limits:
  memory:  5734MiRequests:
  cpu:     4
  memory:  1024Mi


Thanks

Reply via email to