I agree with Robert. Looks like a bug in Flink. Maybe an off-by-one issue (violating index is 32768 and the default memory segment size is 32KB).
Which Flink version are you using? In case you are using a custom build, can you share the commit ID (is reported in the first lines of the JobManager log file)? Thanks, Fabian 2015-11-10 18:29 GMT+01:00 Robert Metzger <rmetz...@apache.org>: > Hi Ali, > > this could be a bug in Flink. > Can you share the code of your program with us to debug the issue? > > On Tue, Nov 10, 2015 at 6:25 PM, Kashmar, Ali <ali.kash...@emc.com> wrote: > > > Hello, > > > > I’m getting this error while running a streaming module on a cluster of 3 > > nodes: > > > > > > java.lang.ArrayIndexOutOfBoundsException: 32768 > > > > at org.apache.flink.core.memory.MemorySegment.get(MemorySegment.java:178) > > > > at > > > org.apache.flink.runtime.io.network.api.serialization.SpillingAdaptiveSpanningRecordDeserializer$NonSpanningWrapper.readByte(SpillingAdaptiveSpanningRecordDeserializer.java:214) > > > > at > > > org.apache.flink.runtime.io.network.api.serialization.SpillingAdaptiveSpanningRecordDeserializer$NonSpanningWrapper.readUnsignedByte(SpillingAdaptiveSpanningRecordDeserializer.java:219) > > > > at org.apache.flink.types.StringValue.readString(StringValue.java:764) > > > > at > > > org.apache.flink.api.common.typeutils.base.StringSerializer.deserialize(StringSerializer.java:68) > > > > at > > > org.apache.flink.api.common.typeutils.base.StringSerializer.deserialize(StringSerializer.java:73) > > > > at > > > org.apache.flink.api.common.typeutils.base.StringSerializer.deserialize(StringSerializer.java:28) > > > > at > > > org.apache.flink.api.java.typeutils.runtime.PojoSerializer.deserialize(PojoSerializer.java:499) > > > > at > > > org.apache.flink.streaming.runtime.streamrecord.StreamRecordSerializer.deserialize(StreamRecordSerializer.java:102) > > > > at > > > org.apache.flink.streaming.runtime.streamrecord.StreamRecordSerializer.deserialize(StreamRecordSerializer.java:29) > > > > at > > > org.apache.flink.runtime.plugable.ReusingDeserializationDelegate.read(ReusingDeserializationDelegate.java:57) > > > > at > > > org.apache.flink.runtime.io.network.api.serialization.SpillingAdaptiveSpanningRecordDeserializer.getNextRecord(SpillingAdaptiveSpanningRecordDeserializer.java:110) > > > > at > > > org.apache.flink.streaming.runtime.io.StreamingAbstractRecordReader.getNextRecord(StreamingAbstractRecordReader.java:80) > > > > at > > > org.apache.flink.streaming.runtime.io.StreamingMutableRecordReader.next(StreamingMutableRecordReader.java:36) > > > > at > > > org.apache.flink.runtime.operators.util.ReaderIterator.next(ReaderIterator.java:59) > > > > at > > > org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.readNext(OneInputStreamTask.java:68) > > > > at > > > org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.invoke(OneInputStreamTask.java:101) > > > > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559) > > > > at java.lang.Thread.run(Thread.java:745) > > > > > > Here’s the configuration for each node: > > > > > > jobmanager.heap.mb: 2048 > > > > taskmanager.heap.mb: 4096 > > > > taskmanager.numberOfTaskSlots: 5 > > > > > > I’m not even sure where to start with this one so any help is > appreciated. > > > > > > Thanks, > > > > Ali > > >
