Re: Clarification regarding BlockTree implementation of IntersectTermsEnum

2019-04-04 Thread Stamatis Zampetakis
I am mostly referring to keyset pagination so sorting the field is always necessary. For the same reason range queries are inevitable. Στις Τρί, 2 Απρ 2019 στις 5:08 μ.μ., ο/η Robert Muir έγραψε: > I think instead of using range queries for custom pagination, it would > be best to simply sort by

RE: Why does Lucene 7.4.0 commit() Increase Memory Usage x2

2019-04-04 Thread Uwe Schindler
Small correction: It's not fully true that the JVM "never" gives back memory to the operating system: The G1 collector can give back memory to the OS since the beginning, but it does this only on full GCs which it tries to prevent. But: The default collector as shipped with Java 8 (ParallelGC)

RE: Why does Lucene 7.4.0 commit() Increase Memory Usage x2

2019-04-04 Thread Uwe Schindler
Hi, Thanks Adrien. With current JVM versions (Java 8 or Java 11), the garbage collector never gives back memory to the operating system, once it has allocated that. Due to now heavy usage of containers and similar techniques, there are efforts on the JVM front to change that: At least the G1 ga

Re: Why does Lucene 7.4.0 commit() Increase Memory Usage x2

2019-04-04 Thread Adrien Grand
I think what you are experiencing is just due to how the JVM works: it happily reserves memory to the operating system if it thinks it might need it, and then it's reluctant to give it back because it assumes that if it has needed so much memory in the past, it might need it again in the future. If

Re: Lucene migrate to 6.6.5 from 5.5.3

2019-04-04 Thread Adrien Grand
Difference in performance, indexing speed and index size will greatly depend on the data. There is some data at https://www.elastic.co/blog/searching-numb3rs-in-5.0. FYI there is one known case when points perform worse than legacy numeric fields: exact queries on low-cardinality fields. But if yo