Re: RDD block has negative value in Spark UI

Already a know minor issue https://issues.apache.org/jira/plugins/servlet/mobile#issue/SPARK-10141 On Wed, 7 Dec 2022, 15:09 K B M Kaala Subhikshan, < kbmkaalasubhiks...@gmail.com> wrote: > Could you explain why the RDD block has a negative value? > >

[SPARK Memory management] Does Spark support setting limits/requests for driver/executor memory ?

Hello, We are running Spark on Kubernetes and noticed that driver/executors use the same value for memory request and memory limit. We see that limits/requests can be set only for cpu using the following options: spark.kubernetes.{driver/executor}.limit.cores and spark.kubernetes.{driver/executor}