Re: OOM error

2016-12-23 Thread Uwe Schindler
Hi, For sorting you should add the fields as DocValues type in addition to indexed fields (if you also query on them). That ist the problem. But for that you should update your Lucene version. For that reason Lucene 6 no longer has FieldCache, which emulates DocValues loading everything to hea

Re: Sorting, Range Query, faceting - NumericDocValuesField Vs LongField

2016-12-23 Thread Erick Erickson
And I frequently forget which list I'm on. Siiigggh... Listen to Mike ;) Best, Erick On Fri, Dec 23, 2016 at 3:40 AM, Kumaran Ramasubramanian wrote: > Thanks Erick and Mike. i am using lucene 4.10.4 directly. > > > i have observed better performance in LongField compared to lexicographic > sort

OOM error

2016-12-23 Thread Sadaf
Hi, This is the index we are using: Number of fields: 355. Number of documents: 225 thousand. Number of terms: 5522 thousand. The index size is around 800MB. TermInfos index divisor: 1 Index format: Lucene 4.0. We are getting Java OutOfMemory error with searches. We are using a heap size of 1

Re: CompressingStoredFieldReader object taking lot of heap space

2016-12-23 Thread Adrien Grand
I don't know anything about JBoss but I am pretty sure this is something that can be configured. A quick Google search suggests the default thread pool size is 512 * number_of_processors, which is indeed likely too high for Lucene: https://developer.jboss.org/thread/267570. Looks like it can be con

RE: CompressingStoredFieldReader object taking lot of heap space

2016-12-23 Thread Mukul Ranjan
Thanks Adrien for quick reply! We use jboss app server to handle application level threads. Thanks, Mukul Ranjan From: Adrien Grand [mailto:jpou...@gmail.com] Sent: Friday, December 23, 2016 6:33 PM To: Mukul Ranjan ; Lucene Users Mailing List Subject: Re: CompressingStoredFieldReader object t

Re: CompressingStoredFieldReader object taking lot of heap space

2016-12-23 Thread Adrien Grand
Hi, SearcherManager only cares about managing readers/searchers in a thread-safe way, it does not know about the thread pool it runs in. How does your application handle threads? I'm thinking you might be running in a servlet container, or a similar framework that handles it for you? Le ven. 23

Re: Sorting, Range Query, faceting - NumericDocValuesField Vs LongField

2016-12-23 Thread Kumaran Ramasubramanian
Thanks Erick and Mike. i am using lucene 4.10.4 directly. i have observed better performance in LongField compared to lexicographic sorting. i can understand, it is due to trie structure of LongField, But one more doubt, Will uninversion process happen in IntField / LongField too? Thanks for th

Re: Sorting, Range Query, faceting - NumericDocValuesField Vs LongField

2016-12-23 Thread Michael McCandless
Note that Erick is giving you the Solr syntax below, but if you are using Lucene directly, that obviously doesn't apply (though the same general concepts do). I would strongly recommend not using uninversion: it's an archaic and costly option that Lucene only offered long ago because it didn't hav