Re: Term Dictionary taking up lots of memory, looking for solutions, lucene 5.3.1

2017-05-17 Thread Adrien Grand
Is upgrading to Lucene 6 and using points rather than terms an option? Points typically have lower memory usage (see GeoPoint which is based on terms vs LatLonPoint which is based on points at http://people.apache.org/~mikemccand/geobench.html#reader-heap). Le jeu. 18 mai 2017 à 02:35, Tom Hirschf

Term Dictionary taking up lots of heap memory, looking for solutions, lucene 5.3.1

2017-05-17 Thread Tom Hirschfeld
Hey! I am working on a lucene based service for reverse geocoding. We have a large index with lots of unique terms (550 million) and it appears that we're running into issue with memory on our leaf servers as the term dictionary for the entire index is being loaded into heap space. If we allocate

Term Dictionary taking up lots of memory, looking for solutions, lucene 5.3.1

2017-05-17 Thread Tom Hirschfeld
Hey! I am working on a lucene based service for reverse geocoding. We have a large index with lots of unique terms (550 million) and it appears that we're running into issue with memory on our leaf servers as the term dictionary for the entire index is being loaded into heap space. If we allocate