Hi Spark Experts,
Today I tried Spark 2.0 on YARN and also enabled
Dynamic Resource Allocation feature, I just find that no matter I specify
--num-executor in spark-submit command or not, the Dynamic Resource Allocation
is used, but I remember when I specify --num-executor option in spark-submit
command in Spark 1.6, the Dynamic Resource Allocation feature will not be
used/effect for that job. And I can see below log in Spark 1.6 .
Is this a behavior change in Spark 2.0? And How can I
disable Dynamic Resource Allocation for a specific job submission temporarily
as before?
