Re: Memory segment error

2015-03-30 Thread Fabian Hueske
gt; > > Andra > > > > > > > > > > > > > > On Fri, Mar 27, 2015 at 12:08 AM, Andra Lungu < > > > lungu.an...@gmail.com > > > > > > > > > > > > wrote: > > > > > > > > > > &g

Re: Memory segment error

2015-03-30 Thread Andra Lungu
> > > > > > > > > > > On Fri, Mar 27, 2015 at 12:02 AM, Andra Lungu < > > > lungu.an...@gmail.com > > > > > > > > > > > > wrote: > > > > > > > > > > > > > >&

Re: Memory segment error

2015-03-30 Thread Fabian Hueske
> > wrote: > > > > > > > > > > > >> Hi Fabian, > > > > > >> > > > > > >> I uploaded a file with my execution plan. > > > > > >> > > > > > >> On Thu, Mar 26, 2015 at 11:50 PM, Fabian Hueske

Re: Memory segment error

2015-03-30 Thread Andra Lungu
;>> Since you have 820MB of managed memory and the size of a memory > > page > > > is > > > > >>> 32KB, there should be more than 25k pages available. > > > > >>> > > > > >>> Can you post

Re: Memory segment error

2015-03-30 Thread Fabian Hueske
es available. > > > >>> > > > >>> Can you post the execution plan of the program you execute ( > > > >>> ExecutionEnvironment.getExecutionPlan() )? > > > >>> > > > >>> Best, Fabian > > > >>> &

Re: Memory segment error

2015-03-30 Thread Andra Lungu
nd 5 nodes, that should be more thank enough. > > >>> > > > >>> > On Thu, Mar 26, 2015 at 11:24 PM, Andra Lungu < > lungu.an...@gmail.com > > > > > >>> > wrote: > > >>> > > > >>> > > Sure, >

Re: Memory segment error

2015-03-30 Thread Fabian Hueske
e, > >>> > > > >>> > > 3470 [main] INFO > org.apache.flink.runtime.taskmanager.TaskManager - > >>> > > Using 820 MB for Flink managed memory. > >>> > > > >>> > > On Thu, Mar 26, 2015 at 4:48 PM, Robert Metzger

Re: Memory segment error

2015-03-30 Thread Andra Lungu
5 at 4:48 PM, Robert Metzger >> > >>> > > wrote: >>> > > >>> > >> Hi, >>> > >> >>> > >> during startup, Flink will log something like: >>> > >> 16:48:09,669 INFO org.apache.flink.runtime.task

Re: Memory segment error

2015-03-26 Thread Andra Lungu
>> 16:48:09,669 INFO org.apache.flink.runtime.taskmanager.TaskManager >> > >> - Using 1193 MB for Flink managed memory. >> > >> >> > >> Can you tell us how much memory Flink is managing in your case? >> > >> >> > &g

Re: Memory segment error

2015-03-26 Thread Andra Lungu
09,669 INFO org.apache.flink.runtime.taskmanager.TaskManager > > >> - Using 1193 MB for Flink managed memory. > > >> > > >> Can you tell us how much memory Flink is managing in your case? > > >> > > >> > > >> > > >> On Thu, Mar 26, 201

Re: Memory segment error

2015-03-26 Thread Fabian Hueske
ar 26, 2015 at 4:46 PM, Andra Lungu > >> wrote: > >> > >> > Hello everyone, > >> > > >> > I guess I need to revive this old discussion: > >> > > >> > > >> > http://apache-flink-incubator-mailing-list-archive.1

Re: Memory segment error

2015-03-26 Thread Andra Lungu
managing in your case? >> >> >> >> On Thu, Mar 26, 2015 at 4:46 PM, Andra Lungu >> wrote: >> >> > Hello everyone, >> > >> > I guess I need to revive this old discussion: >> > >> > >> http://apache-flink-incubator

Re: Memory segment error

2015-03-26 Thread Andra Lungu
s old discussion: > > > > > http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/Memory-segment-error-when-migrating-functional-code-from-Flink-0-9-to-0-8-td3687.html > > > > At that point, the fix was to kindly ask Alex to make his project work &

Re: Memory segment error

2015-03-26 Thread Robert Metzger
llo everyone, > > I guess I need to revive this old discussion: > > http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/Memory-segment-error-when-migrating-functional-code-from-Flink-0-9-to-0-8-td3687.html > > At that point, the fix was to kindly ask Alex to

Memory segment error

2015-03-26 Thread Andra Lungu
Hello everyone, I guess I need to revive this old discussion: http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/Memory-segment-error-when-migrating-functional-code-from-Flink-0-9-to-0-8-td3687.html At that point, the fix was to kindly ask Alex to make his project work with

Re: Memory segment error when migrating functional code from Flink 0.9 to 0.8

2015-02-09 Thread Andra Lungu
Hi Stephan, I tried the solution with DeltaIteration#setSolutionSetUnManaged(), unfortunatelly the error is still there... Even when I try to run it with just one iteration... Also, I am not sure that the job can be broken into subparts in this particular case. Any other suggestions would be appr

Re: Memory segment error when migrating functional code from Flink 0.9 to 0.8

2015-02-09 Thread Stephan Ewen
This is actually a problem of the number of memory segments available to the hash table for the solution set. For complex pipelines, memory currently gets too fragmented. There are two workarounds, until we do the dynamic memory management, or break it into shorter pipelines: Break the job up int

Re: Memory segment error when migrating functional code from Flink 0.9 to 0.8

2015-02-09 Thread Till Rohrmann
Hi Andra, have you tried increasing the number of network buffers in your cluster? You can control by the configuration value: taskmanager.network.numberOfBuffers: #numberBuffers Greets, Till On Mon, Feb 9, 2015 at 9:56 AM, Andra Lungu wrote: > Hello everyone, > > I am implementing a graph a

Memory segment error when migrating functional code from Flink 0.9 to 0.8

2015-02-09 Thread Andra Lungu
Hello everyone, I am implementing a graph algorithm as part of a course and I will also add it to the Flink- Gelly examples. My problem is that I started developing it in the Gelly repository, which runs on flink 0.9. It works like a charm there, but in order to test in on a cluster to see its rea