Hello,
We are using lucene 2.9.0. and ran into OutOfMemory error when sorting on a highly unique field on a big index. After doing some research we learned that lucene will load the sort field value for all documents into memory to do sorting, and ended up with the OutOfMemory if the index is too big. The problem is no matter how much heap size you allocated for jvm, you will run into this problem eventually when your index is big enough and the sort field value is unique enough regardless how small amount of documents actually match the query ... This is a bit ... annoying. I only want to sort the 10 docs that match the query, why I need to load millions of field values into memory? Then we are thinking to disable the sort from lucene search and do the sorting ourselves. that is, after getting all matching docs from search(since we know the matching docs won't be too much), we do the sorting "manually". This requires us write our own sorting methods and it needs to support multiple field sort. so my question is is this the right way to go? or maybe lucene/solr already has the similiar thing? Thank you very much, -Fujian -- View this message in context: http://lucene.472066.n3.nabble.com/is-this-the-right-way-to-go-tp883464p883464.html Sent from the Lucene - Java Users mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
