Re: EOF exception from ramDirectory search in spark

2018-05-11 Thread Tom Hirschfeld
org.apache.lucene.util.packed.DirectReader$DirectPackedReader48.get(DirectReader.java:305) ... 35 more On Fri, May 11, 2018 at 1:15 AM, Adrien Grand wrote: > Can you share the full stack trace? > > Le ven. 11 mai 2018 à 04:19, Tom Hirschfeld a > écrit : > > > Hey All, > > I h

EOF exception from ramDirectory search in spark

2018-05-10 Thread Tom Hirschfeld
to address this issue but I have been unable to find out whats going on. Any hint as to what might be happening here? Best, Tom Hirschfeld

Lucene, Spark, HDFS question

2018-03-13 Thread Tom Hirschfeld
s this compatible? Are we able to store our index in HDFS and read from a spark job? Best, Tom Hirschfeld

NumericDocValues vs SortedNumericDocValues

2018-02-05 Thread Tom Hirschfeld
sort about 200 results. My specific questions are, for our use case, how do these two fields differ in: 1) total index size 2) query time performance/impact on sorting 3) any other "gotchas" I may not have thought of yet Thanks for your time & assistance! Best, Tom Hirschfeld

Spatial Indexing of Polygons

2017-08-14 Thread Tom Hirschfeld
ed if it exists. Is there a recommended way to support indexing and searching of polygons (building footprint sized polygons, not huge ones)? If so what is the currently recommended API to use? We are currently thinking about using the s2cell library from google. Best, Tom Hirschfeld

Optimizing number of segments in lucene index (no writes/deletes, only reads)

2017-06-13 Thread Tom Hirschfeld
segment per cpu in prod? 1 segment per core in prod? Something else? Best, Tom Hirschfeld

Re: Term Dictionary taking up lots of memory, looking for solutions, lucene 5.3.1

2017-06-13 Thread Tom Hirschfeld
Once again, thanks for your help. Best, Tom Hirschfeld On Thu, May 18, 2017 at 4:22 AM, Uwe Schindler wrote: > Hi, > Are you sure that the term index is the problem? Even with huge indexes > you never need 65 good of heap! That's impossible. > Are you sure that your problem is not

Term Dictionary taking up lots of heap memory, looking for solutions, lucene 5.3.1

2017-05-17 Thread Tom Hirschfeld
is issue? If so, how do I got about loading an alternative codec and configuring it to my needs? I'm having trouble finding docs/examples of how this is used in the real world so even if you point me to a repo or docs somewhere I'd appreciate it. Thanks! Best, Tom Hirschfeld

Term Dictionary taking up lots of memory, looking for solutions, lucene 5.3.1

2017-05-17 Thread Tom Hirschfeld
? If so, how do I got about loading an alternative codec and configuring it to my needs? I'm having trouble finding docs/examples of how this is used in the real world so even if you point me to a repo or docs somewhere I'd appreciate it. Thanks! Best, Tom Hirschfeld