Fetching stored data takes more time

2014-07-30 Thread Ganesh
Hello all, I am using Lucene 4.9 and the index size is 7 GB. Search is faster, it takes 1 second to return the results (50 hits). I loop through the result and fetching the stored data for all and it takes more time. Some times it takes more than a minute. Could some one guide me.. how to r

2.9.2 Memory issue 8.0GB or more / OOM with Term / Highlighting

2014-07-30 Thread Baldwin, David
I am looking to track down an issue in 2.9.2 where during highlighting, certain data may cause rapid memory usage and OOM exception in java: --- java.lang.OutOfMemoryError: Java heap space at org.apache.lucene.analysis.Token.growTermBuffer(Token.java:470) at

Re: How to get the Terms from the TermsEnum of an IntField ?

2014-07-30 Thread Olivier Binda
On 07/30/2014 11:31 AM, Michael McCandless wrote: If you indexed with precStep=Integer.MAX_VALUE then your index should only have the "shift 0" terms, I think. Can you boil the exception case (values 1 to 2500) down to a small test case? The issue was the copyChars of the BytesRef class /**

Is it right to index multi-valued fields with hash value?

2014-07-30 Thread Kumaran Ramasubramanian
Hi All During search, i find it difficult to handle every multi-valued field with different analyzers. So i believe indexing multi-valued field with hash value may solve the problem of searching with different analyzers. Any of you tried hash value in lucene index? if so, please share limit

Re: How to get the Terms from the TermsEnum of an IntField ?

2014-07-30 Thread Olivier Binda
On 07/30/2014 11:31 AM, Michael McCandless wrote: If you indexed with precStep=Integer.MAX_VALUE then your index should only have the "shift 0" terms, I think. Damn :( I was wondering if that was the case but I wasn't sure because of (eventual compression and) the trie structure. This mean

Re: How to get the Terms from the TermsEnum of an IntField ?

2014-07-30 Thread Michael McCandless
If you indexed with precStep=Integer.MAX_VALUE then your index should only have the "shift 0" terms, I think. Can you boil the exception case (values 1 to 2500) down to a small test case? Mike McCandless http://blog.mikemccandless.com On Wed, Jul 30, 2014 at 2:21 AM, Olivier Binda wrote: > He