You really should be using the same IndexSearcher for successive
searches. Sorting works best when done with a "warm" searcher. Have
a look at Solr's warming strategy, and consider adopting that in some
way.
Erik
On Oct 12, 2006, at 3:04 PM, <[EMAIL PROTECTED]> wrote:
Hi folks,
I am using Lucene 2.0
In our application, I am indexing a stream of documents. Each
document is fairly small (< 1 KB), but there can be 10's of
millions of documents. Each document has a Timestamp field. Users
can enter free-form searches and a date/time range. They are most
interested in the most recent documents (as indicated in the
Timestamp field). An obvious way to do achieve this is to
searcher = new IndexSearcher(indexDir);
RangeFilter rf = new RangeFilter("day", start, end, true, true);
hits = searcher.search(query,rf,new Sort(new SortField[]{
new SortField
("timestamp",SortField.STRING,true )}));
Depending on the query, there may be millions of hits results. If
the same query is executed several times in quick succession, the
heap quickly runs out of memory. I suspect that this is because
Lucene needs to load all the millions of hits in order to sort the
results.
My idea is to avoid the Sort() entirely. Is there a way, during
indexing (or by setting Weights inside the query) to automatically
set the score for more recent documents higher?
Thanks
--
Solidguy
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
