Hi I am using the below command to run a spark job and I get an error like "Container preempted by scheduler"
I am not sure if it's related to the wrong usage of Memory: nohup ~/spark1.3/bin/spark-submit \ --num-executors 50 \ --master yarn \ --deploy-mode cluster \ --queue adhoc \ --driver-memory 18G \ --executor-memory 12G \ --class main.ru.<custom>.bigdata.externalchurn.Main \ --conf "spark.task.maxFailures=100" \ --conf "spark.yarn.max.executor.failures=10000" \ --conf "spark.executor.cores=1" \ --conf "spark.akka.frameSize=50" \ --conf "spark.storage.memoryFraction=0.5" \ --conf "spark.driver.maxResultSize=10G" \ ~/external-flow/externalChurn-1.0-SNAPSHOT-shaded.jar \ prepareTraining=true \ prepareTrainingMNP=true \ prepareMap=false \ bouldozerMode=true \ &> ~/external-flow/run.log & echo "STARTED" tail -f ~/external-flow/run.log Thanks,
