Hi Jamie, thanks for reporting back the numbers about your usage of NumericField and NumericRangeQuery! I am glad to hear about it.
> Sure. As soon as I get access to the server again, I'll get the mem > stats for you. I will say that Lucene was consuming a large amount of > memory before we moved over to using Numerics. The reason for this is > that we were encoding dates as strings. Our date time strings were > unique, so as the number of records exploded, so too did the number of > terms in the index. As I understand it (and I am no Lucene expert), > Lucene's sorting mechanisms need to load up all the terms in the index > in memory during a sort. Thus, if you execute a sorted search, Lucene's > memory consumption goes through the roof. Using Numerics avoids this > problem. Thats true, as you only need to store an integer per document in the cache (and not a String). Also performance for FieldCache warmup and sorting is higher. > There are lot of other strategies we used to reduce memory consumption. > Like, making sure that you are caching Searchers and IndexReaders, etc. > > Regards, > > Jamie > > > > On 2010/03/19 07:04 PM, Monique Monteiro wrote: > > Hi Jamie, > > > > could you please tell us how much memory does your application > consume > > with Lucene? I'm asking it because we are having memory consumption > problems > > with a 32GB index and 1.5GB od RAM allocated to our web application. > At the > > momento, we use textual search. > > Thanks in advance, > > Monique > > On Fri, Mar 19, 2010 at 8:49 AM, Jamie<ja...@stimulussoft.com> > wrote: > > > > > >> Hi Guys > >> > >> I just wanted to congratulate the Lucene guys for a fine job on > 3.0!! > >> > >> Since we switched our indexes to using integer based range queries > based on > >> Date (YYMMHHSS), search speed is lightening fast and memory > consumption has > >> dropped considerably! > >> > >> Some stats: > >> > >> Indexed Docs: 7.2M emails > >> Index Size: 24 GB (non optimized) > >> Search Speed: 0.06 - 0.09 seconds (with sort YYMMHHSS date) > >> > >> Index stored on 4 SAS HDD hitachi RAID 10 > >> 16G RAM > >> 2x Xeon 4 core 2.4Gz > >> OS FreeBSD 7.2 > >> Filesystem UFS2 gjournal > >> > >> I believe we are using all search performance recommendations now. > >> > >> Good job! > >> > >> Jamie > >> > >> > >> > >> -------------------------------------------------------------------- > - > >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > >> For additional commands, e-mail: java-user-h...@lucene.apache.org > >> > >> > >> > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
