Re: Running Spark und YARN on AWS EMR

2017-07-17 Thread Takashi Sasaki
sification": "spark-defaults", > "Properties": { > "spark.default.parallelism": "1000" > } > } > > Good luck, and I hope this is helpful! > > --Josh > > > On Mon, Jul 17, 2017 at 4:59 PM, Takashi Sasaki

Re: Running Spark und YARN on AWS EMR

2017-07-17 Thread Takashi Sasaki
Hi Pascal, The error also occurred frequently in our project. As a solution, it was effective to specify the memory size directly with spark-submit command. eg. spark-submit executor-memory 2g Regards, Takashi > 2017-07-18 5:18 GMT+09:00 Pascal Stammer : >> Hi, >> >> I am running a Spark 2.1