Hi,
I tested three configurations:
taskmanager.heap.mb: 6144, taskmanager.memory.fraction: 0.5
taskmanager.heap.mb: 5544, taskmanager.memory.fraction: 0.6
taskmanager.heap.mb: 5144, taskmanager.memory.fraction: 0.7
The error occurs in all three configurations.
In the last configuration, I can even find another exception in the logs of
one of the taskmanagers:
19.Apr. 13:39:29 INFO Task -
IterationHead(WorksetIteration (Resolved-Redirects)) (10/10) switched to
FAILED : java.lang.IndexOutOfBoundsException: Index: 161, Size: 161
at java.util.ArrayList.rangeCheck(ArrayList.java:635)
at java.util.ArrayList.get(ArrayList.java:411)
at
org.apache.flink.runtime.operators.hash.InMemoryPartition$WriteView.resetTo(InMemoryPartition.java:352)
at
org.apache.flink.runtime.operators.hash.InMemoryPartition$WriteView.access$100(InMemoryPartition.java:301)
at
org.apache.flink.runtime.operators.hash.InMemoryPartition.appendRecord(InMemoryPartition.java:226)
at
org.apache.flink.runtime.operators.hash.CompactingHashTable.insertOrReplaceRecord(CompactingHashTable.java:536)
at
org.apache.flink.runtime.operators.hash.CompactingHashTable.buildTableWithUniqueKey(CompactingHashTable.java:347)
at
org.apache.flink.runtime.iterative.task.IterationHeadPactTask.readInitialSolutionSet(IterationHeadPactTask.java:209)
at
org.apache.flink.runtime.iterative.task.IterationHeadPactTask.run(IterationHeadPactTask.java:270)
at
org.apache.flink.runtime.operators.RegularPactTask.invoke(RegularPactTask.java:362)
at
org.apache.flink.runtime.execution.RuntimeEnvironment.run(RuntimeEnvironment.java:217)
at java.lang.Thread.run(Thread.java:745)
Greetings
Stefan
On 17 April 2015 at 16:05, Stephan Ewen <[email protected]> wrote:
> Hi!
>
> After a quick look over the code, this seems like a bug. One cornercase of
> the overflow handling code does not check for the "running out of memory"
> condition.
>
> I would like to wait if Robert Waury has some ideas about that, he is the
> one most familiar with the code.
>
> I would guess, though, that you should be able to work around that by
> either setting the solution set as "unmanaged", or by slightly changing the
> memory configuration. It seems a rare cornercase, only if the memory runs
> out in a very special situation. You may be able to avoid running into it
> when using slightly more or less memory.
>
> Greetings,
> Stephan
>
>
> On Fri, Apr 17, 2015 at 3:59 PM, Stefan Bunk <[email protected]>
> wrote:
>
>> Hi Squirrels,
>>
>> I have some trouble with a delta-iteration transitive closure program [1].
>> When I run the program, I get the following error:
>>
>> java.io.EOFException
>> at
>> org.apache.flink.runtime.operators.hash.InMemoryPartition$WriteView.nextSegment(InMemoryPartition.java:333)
>> at
>> org.apache.flink.runtime.memorymanager.AbstractPagedOutputView.advance(AbstractPagedOutputView.java:140)
>> at
>> org.apache.flink.runtime.memorymanager.AbstractPagedOutputView.writeByte(AbstractPagedOutputView.java:223)
>> at
>> org.apache.flink.runtime.memorymanager.AbstractPagedOutputView.writeLong(AbstractPagedOutputView.java:291)
>> at
>> org.apache.flink.runtime.memorymanager.AbstractPagedOutputView.writeDouble(AbstractPagedOutputView.java:307)
>> at
>> org.apache.flink.api.common.typeutils.base.DoubleSerializer.serialize(DoubleSerializer.java:62)
>> at
>> org.apache.flink.api.common.typeutils.base.DoubleSerializer.serialize(DoubleSerializer.java:26)
>> at
>> org.apache.flink.api.scala.typeutils.CaseClassSerializer.serialize(CaseClassSerializer.scala:89)
>> at
>> org.apache.flink.api.scala.typeutils.CaseClassSerializer.serialize(CaseClassSerializer.scala:29)
>> at
>> org.apache.flink.runtime.operators.hash.InMemoryPartition.appendRecord(InMemoryPartition.java:219)
>> at
>> org.apache.flink.runtime.operators.hash.CompactingHashTable.insertOrReplaceRecord(CompactingHashTable.java:536)
>> at
>> org.apache.flink.runtime.operators.hash.CompactingHashTable.buildTableWithUniqueKey(CompactingHashTable.java:347)
>> at
>> org.apache.flink.runtime.iterative.task.IterationHeadPactTask.readInitialSolutionSet(IterationHeadPactTask.java:209)
>> at
>> org.apache.flink.runtime.iterative.task.IterationHeadPactTask.run(IterationHeadPactTask.java:270)
>> at
>> org.apache.flink.runtime.operators.RegularPactTask.invoke(RegularPactTask.java:362)
>> at
>> org.apache.flink.runtime.execution.RuntimeEnvironment.run(RuntimeEnvironment.java:217)
>> at java.lang.Thread.run(Thread.java:745)
>>
>> Both input files have been generated and written to HDFS by Flink jobs. I
>> already ran the Flink program that generated them several times: the error
>> persists.
>> You can find the logs at [2] and [3].
>>
>> I am using the 0.9.0-milestone-1 release.
>>
>> Best,
>> Stefan
>>
>>
>> [1] Flink job: https://gist.github.com/knub/0bfec859a563009c1d57
>> [2] Job manager logs: https://gist.github.com/knub/01e3a4b0edb8cde66ff4
>> [3] One task manager's logs:
>> https://gist.github.com/knub/8f2f953da95c8d7adefc
>>
>
>