Hi Karthik ,
You must set the value before the SparkContext (sc) is created. Also don't
assign too much overhead like 20g for maxResultSize , You can set it to 2G
maximum as per your error message.
Also if you are using Java 1.8 , Please add the below section in your
Yarn-site.xml
<property>
<name>yarn.nodemanager.vmem-pmem-ratio</name>
<value>5</value>
</property>
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Saprk-1-6-Driver-Memory-Issue-tp27063p27064.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
