Re: EOF exception from ramDirectory search in spark

2018-05-11 Thread Tom Hirschfeld
Here is an example of a full stack trace java.lang.RuntimeException: java.io.EOFException: seek beyond EOF: pos=3080604 vs length=533151: RAMInputStream(name=RAMInputStream(name=_cw4e_Lucene54_0.dvd) [slice=randomaccess]) at org.apache.lucene.util.packed.DirectReader$DirectPackedReader48.

Re: EOF exception from ramDirectory search in spark

2018-05-11 Thread Adrien Grand
Can you share the full stack trace? Le ven. 11 mai 2018 à 04:19, Tom Hirschfeld a écrit : > Hey All, > I have a fun issue I'm dealing with at the junction of lucene and spark. > > I have an RDD[(key, iterator1, iterator2)] > > I run a mapPartitions on the RDD, and for each partition, I create a