data size: text file, 315G
cmd:
./spark-submit --class com.spark.test.JavaWordCountWithSave --num-executors 7
--executor-memory 60g --driver-memory 2g --executor-cores 32
--master yarn-client /home/cjs/spark-test.jar hdfs://wordcount/input
hdfs://wordcount/output
code of JavaWordCountWithSave:
`
submit 12 spark applications in the same time. yarn web page shows: two task
fail.
the cmd:
./spark-submit--class org.apache.spark.examples.JavaWordCount --master
yarn-cluster ---executor-memory 2g ../lib/spark-examples_2.10-1.1.0.jar
hdfs://hacluster/bigData
driver log of one fa