Hello,
We are running Spark on Kubernetes and noticed that driver/executors use
the same value for memory request and memory limit. We see that
limits/requests can be set only for cpu using the following options:
spark.kubernetes.{driver/executor}.limit.cores and
spark.kubernetes.{driver/executor}.request.cores.
In our case, it is useful to set a memory request lower than the memory
limit as it helps us better manage our cluster resources.
Is there a way to define limits/requests for memory resources for
driver/executor pods ?
Thanks in advance.
Yosr
