Re: Java heap space and spark.akka.frameSize Inbox x

2014-04-21 Thread Arpit Tak
Also check out this post http://apache-spark-user-list.1001560.n3.nabble.com/Spark-program-thows-OutOfMemoryError-td4268.html On Mon, Apr 21, 2014 at 11:49 AM, Akhil Das wrote: > Hi Chieh, > > You can increase the heap size by exporting the java options (See below, > will increase the heap size

Re: Java heap space and spark.akka.frameSize Inbox x

2014-04-20 Thread Akhil Das
Hi Chieh, You can increase the heap size by exporting the java options (See below, will increase the heap size to 10Gb) export _JAVA_OPTIONS="-Xmx10g" On Mon, Apr 21, 2014 at 11:43 AM, Chieh-Yen wrote: > Can anybody help me? > Thanks. > > Chieh-Yen > > > On Wed, Apr 16, 2014 at 5:18 PM, Chie

Re: Java heap space and spark.akka.frameSize Inbox x

2014-04-20 Thread Chieh-Yen
Can anybody help me? Thanks. Chieh-Yen On Wed, Apr 16, 2014 at 5:18 PM, Chieh-Yen wrote: > Dear all, > > I developed a application that the message size of communication > is greater than 10 MB sometimes. > For smaller datasets it works fine, but fails for larger datasets. > Please check the er

Java heap space and spark.akka.frameSize Inbox x

2014-04-16 Thread Chieh-Yen
Dear all, I developed a application that the message size of communication is greater than 10 MB sometimes. For smaller datasets it works fine, but fails for larger datasets. Please check the error message following. I surveyed the situation online and lots of people said the problem can be solve