Thanks Tristan for sharing this. Actually this happens when I am reading a csv file of 3.5 GB.
best, /Shahab On Tue, May 5, 2015 at 9:15 AM, Tristan Blakers <tris...@blackfrog.org> wrote: > Hi Shahab, > > I’ve seen exceptions very similar to this (it also manifests as negative > array size exception), and I believe it’s a really bug in Kryo. > > See this thread: > > http://mail-archives.us.apache.org/mod_mbox/spark-user/201502.mbox/%3ccag02ijuw3oqbi2t8acb5nlrvxso2xmas1qrqd_4fq1tgvvj...@mail.gmail.com%3E > > Manifests in all of the following situations when working with an object > graph in excess of a few GB: Joins, Broadcasts, and when using the hadoop > save APIs. > > Tristan > > > On 3 May 2015 at 07:26, Olivier Girardot <ssab...@gmail.com> wrote: > >> Can you post your code, otherwise there's not much we can do. >> >> Regards, >> >> Olivier. >> >> Le sam. 2 mai 2015 à 21:15, shahab <shahab.mok...@gmail.com> a écrit : >> >>> Hi, >>> >>> I am using sprak-1.2.0 and I used Kryo serialization but I get the >>> following excepton. >>> >>> java.io.IOException: com.esotericsoftware.kryo.KryoException: >>> java.lang.IndexOutOfBoundsException: Index: 3448, Size: 1 >>> >>> I do apprecciate if anyone could tell me how I can resolve this? >>> >>> best, >>> /Shahab >>> >> >
