Re: GraphFrames' ConnectedComponentSuite test 'two components and two dangling vertices' fails with OutOfMemoryError: Java heap space

ssell.jur...@gmail.com>) escribió: >>>> >>>>> Friends of GraphFrames (github.com/graphframes/graphframes), I have a >>>>> question for you... >>>>> >>>>> I can't get the unit test 'two components and two dangling vertices' >>&g

Re: GraphFrames' ConnectedComponentSuite test 'two components and two dangling vertices' fails with OutOfMemoryError: Java heap space

wo dangling vertices' >>>> in the org.graphframes.lib.ConnectedComponentsSuite >>>> <https://github.com/graphframes/graphframes/blob/649094caf58cfda0eea3e8cd66785aa38104d771/src/test/scala/org/graphframes/lib/ConnectedComponentsSuite.scala#L138-L148> >>>

Re: GraphFrames' ConnectedComponentSuite test 'two components and two dangling vertices' fails with OutOfMemoryError: Java heap space

t test 'two components and two dangling vertices' in >>> the org.graphframes.lib.ConnectedComponentsSuite >>> <https://github.com/graphframes/graphframes/blob/649094caf58cfda0eea3e8cd66785aa38104d771/src/test/scala/org/graphframes/lib/ConnectedComponentsSuite.scala#L138-L148> >>>

Re: GraphFrames' ConnectedComponentSuite test 'two components and two dangling vertices' fails with OutOfMemoryError: Java heap space

8cd66785aa38104d771/src/test/scala/org/graphframes/lib/ConnectedComponentsSuite.scala#L138-L148> >> to pass. It fails with an 'OutOfMemoryError: Java heap space' error. I am a >> little stuck on completing a docs release with a motif finding tutorial >> <https

Re: GraphFrames' ConnectedComponentSuite test 'two components and two dangling vertices' fails with OutOfMemoryError: Java heap space

f58cfda0eea3e8cd66785aa38104d771/src/test/scala/org/graphframes/lib/ConnectedComponentsSuite.scala#L138-L148> > to pass. It fails with an 'OutOfMemoryError: Java heap space' error. I am a > little stuck on completing a docs release with a motif finding tutorial > <https://github

GraphFrames' ConnectedComponentSuite test 'two components and two dangling vertices' fails with OutOfMemoryError: Java heap space

s/graphframes/blob/649094caf58cfda0eea3e8cd66785aa38104d771/src/test/scala/org/graphframes/lib/ConnectedComponentsSuite.scala#L138-L148> to pass. It fails with an 'OutOfMemoryError: Java heap space' error. I am a little stuck on completing a docs release with a motif finding tutorial <https://github.com/graphframes/graph

Re: OutofMemoryError: Java heap space

Thanks, Kelvin :) The error seems to disappear after I decreased both spark.storage.memoryFraction and spark.shuffle.memoryFraction to 0.2 And, some increase on driver memory. Best, Yifan LI > On 10 Feb 2015, at 18:58, Kelvin Chu <2dot7kel...@gmail.com> wrote: > > Since the stacktrace

Re: OutofMemoryError: Java heap space

Since the stacktrace shows kryo is being used, maybe, you could also try increasing spark.kryoserializer.buffer.max.mb. Hope this help. Kelvin On Tue, Feb 10, 2015 at 1:26 AM, Akhil Das wrote: > You could try increasing the driver memory. Also, can you be more specific > about the data volume?

Re: OutofMemoryError: Java heap space

Yes, I have read it, and am trying to find some way to do that… Thanks :) Best, Yifan LI > On 10 Feb 2015, at 12:06, Akhil Das wrote: > > Did you have a chance to look at this doc > http://spark.apache.org/docs/1.2.0/tuning.html > > > Than

Re: OutofMemoryError: Java heap space

Did you have a chance to look at this doc http://spark.apache.org/docs/1.2.0/tuning.html Thanks Best Regards On Tue, Feb 10, 2015 at 4:13 PM, Yifan LI wrote: > Hi Akhil, > > Excuse me, I am trying a random-walk algorithm over a not that large > graph(~1GB raw dataset, including ~5million vertic

Re: OutofMemoryError: Java heap space

Hi Akhil, Excuse me, I am trying a random-walk algorithm over a not that large graph(~1GB raw dataset, including ~5million vertices and ~60million edges) on a cluster which has 20 machines. And, the property of each vertex in graph is a hash map, of which size will increase dramatically during

Re: OutofMemoryError: Java heap space

You could try increasing the driver memory. Also, can you be more specific about the data volume? Thanks Best Regards On Mon, Feb 9, 2015 at 3:30 PM, Yifan LI wrote: > Hi, > > I just found the following errors during computation(graphx), anyone has > ideas on this? thanks so much! > > (I think

OutofMemoryError: Java heap space

Hi, I just found the following errors during computation(graphx), anyone has ideas on this? thanks so much! (I think the memory is sufficient, spark.executor.memory 30GB ) 15/02/09 00:37:12 ERROR Executor: Exception in task 162.0 in stage 719.0 (TID 7653) java.lang.OutOfMemoryError: Java hea