Thanks. We could this feature be expected 2.9 OR 3.0? This is a good feature. Mostly Users store sparse data. All records may not have data for all fields. This will reduce the memory consumption to a large extent. In my case almost 30% of records just store information of reference file pointers and it is not part of initial search.
Regards Ganesh ----- Original Message ----- From: "Mark Miller" <markrmil...@gmail.com> To: <java-user@lucene.apache.org> Sent: Monday, July 20, 2009 10:21 PM Subject: Re: Sorting field contating NULL values consumes field cache memory > Right now, you can't really do anything about it. In the future, with the > new FieldCache API that may go in, you could plug in a custom implementation > that makes tradeoffs for a sparse array of some kind. The docid is currently > the index into the array, but with a custom impl you may be able to use a > sparse array object. > Thats a ways off though. > > - Mark > > On Mon, Jul 20, 2009 at 8:38 AM, Ganesh <emailg...@yahoo.co.in> wrote: > >> Any ideas on this?? >> >> Regards >> Ganesh >> >> ----- Original Message ----- >> From: "Ganesh" <emailg...@yahoo.co.in> >> To: <java-user@lucene.apache.org> >> Sent: Friday, July 17, 2009 2:42 PM >> Subject: Sorting field contating NULL values consumes field cache memory >> >> >> I am doing sorting on DateTime with minute resolution. I am having 90 >> million of records and sorting is consuming nearly 500 MB. 30% records are >> not part of primary result set and they don't have sort field. But field >> cache memory (4 * IndexReader.maxDoc() * (# of different fields actually >> used to sort)) is consumed eventhough 30% of records are not part of sort. >> >> I want to avoid the 30% of records not to be loaded in field cache. How >> could i achieve this. Any idea are greatly appreciated? >> >> Regards >> Ganesh >> Send instant messages to your online friends http://in.messenger.yahoo.com >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> >> Send instant messages to your online friends http://in.messenger.yahoo.com >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> >> > > > -- > -- > - Mark > > http://www.lucidimagination.com > Send instant messages to your online friends http://in.messenger.yahoo.com --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
