Files greater than 20 MB not getting Indexed. No files generated except write.lock even after 8-9 minutes.

2013-08-28 Thread Ankit Murarka
Hello all, Faced with a typical issue. I have many files which I am indexing. Problem Faced: a. File having size less than 20 MB are successfully indexed and merged. b. File having size >20MB are not getting INDEXED.. No Exception is being thrown. Only a lock file is being created in the index

Re: Analyzing suggester for many fields

2013-08-28 Thread Goutham Tholpadi
I implemented a simple TermFreqIterator for wrapping Iterator-s from multiple fields, or from multiple AtomicReaders under an IndexReader. It seems to work for me. In case anyone else wants to use a quick-fix, here it is: http://pastebin.com/Hm2zW9xR . Goutham Tholpadi https://sites.google.com/sit

Unifying IDF for unbalanced shards?

2013-08-28 Thread Yu Zhou
Hello, We have a large collection of documents that consists of multiple balanced shards. Now each shard is quickly approaching its limit. Therefore, we would like to explore the possibility of adding unbalanced shards into the mix. However, that means the IDF and Relevance would take a hit. 

Re: Huge FacetArrays while using SortedSetDocValuesAccumulator

2013-08-28 Thread Shai Erera
Oops you're right, it was committed in LUCENE-4985 which will be released in Lucene 4.5. Shai On Wed, Aug 28, 2013 at 6:16 PM, Krishnamurthy, Kannan < kannan.krishnamur...@contractor.cengage.com> wrote: > Thanks for the response. I double checked that > SortedSetDocValuesAccumulator doesn't tak

Re: Huge FacetArrays while using SortedSetDocValuesAccumulator

2013-08-28 Thread Krishnamurthy, Kannan
Thanks for the response. I double checked that SortedSetDocValuesAccumulator doesn't take a FacetArray in its ctor currently in 4.3.0 and 4.4. But FacetAccumulator does take FacetArray in its ctor. Am I missing something here? We have a high traffic application currently doing about 250 searches

Re: Document boosting and native ordering of results

2013-08-28 Thread Michael van Rooyen
Thanks Uwe! I hadn't investigated DocValues fields, but they look like an exciting addition to Lucene and definitely what we need. The FunctionQuery / CustomScoreQuery would be a great solution, but there doesn't seem to be a ValueSource dedicated to DocValues fields and all the field-based va