Hi Suman,
I'll assume that you are using spark submit to run your application. You
can pass the --num-executors flag to ask for more containers. If you want
to allocate more memory for each executor, you may also pass in the
--executor-memory flag (this accepts a string in the format 1g, 512m etc.
Hi,
I am using Spark 1.1.1 on Yarn. When I try to run K-Means, I see from the Yarn
dashboard that only 3 containers are being used. How do I increase the number
of containers used?
P.S: When I run K-Means on Mahout with the same settings, I see that there are
25-30 containers being used.
