I am actually answering myself as I have check on master 3.x branch, and
there is this feature !
https://issues.apache.org/jira/browse/SPARK-27754
So my understanding was correct.
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
-
Hello all,
I am surprised that it is not possible to define
"spark.kubernetes.driver.request.cores" when submitting a spark job on
kubernetes.
My understanding is that it would limit the CPU requests for the driver on
the k8s cluster and we could still define how many cores (threads) we use in
th
Hello Jack,
You can also have a look at “Babar”, there is a nice “flame graph” feature
too. I haven’t had the time to test it out.
https://github.com/criteo/babar
JC
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
--
