Re: spark 1.1.0 requested array size exceed vm limits

2014-09-05 Thread marylucy
I set 200,it remain failed in second step,(map and mapPartition in webui) In spark1.0.2 stable version ,it works well in first step,configuration same as 1.1.0 - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For addit

Re: spark 1.1.0 requested array size exceed vm limits

2014-09-05 Thread Ankur Dave
At 2014-09-05 21:40:51 +0800, marylucy wrote: > But running graphx edgeFileList ,some tasks failed > error:requested array size exceed vm limits Try passing a higher value for minEdgePartitions when calling GraphLoader.edgeListFile. Ankur --

spark 1.1.0 requested array size exceed vm limits

2014-09-05 Thread marylucy
I am building spark-1.0-rc4 with maven,following http://spark.apache.org/docs/latest/building-with-maven.html But running graphx edgeFileList ,some tasks failed error:requested array size exceed vm limits error:executor lost Can anyone know how to fit it