when I submit a job using spark shell I get something like this [Stage 0:========> (36814 + 4) / 220129]
Now all I want is I want to increase number of parallel tasks running from 4 to 16 so I exported an env variable called SPARK_WORKER_CORES=16 in conf/spark-env.sh. I though that should do it but it doesn't. It still shows me 4. any idea? Thanks much!
