Hi Monique
Sure. As soon as I get access to the server again, I'll get the mem
stats for you. I will say that Lucene was consuming a large amount of
memory before we moved over to using Numerics. The reason for this is
that we were encoding dates as strings. Our date time strings were
unique, so as the number of records exploded, so too did the number of
terms in the index. As I understand it (and I am no Lucene expert),
Lucene's sorting mechanisms need to load up all the terms in the index
in memory during a sort. Thus, if you execute a sorted search, Lucene's
memory consumption goes through the roof. Using Numerics avoids this
problem.
There are lot of other strategies we used to reduce memory consumption.
Like, making sure that you are caching Searchers and IndexReaders, etc.
Regards,
Jamie
On 2010/03/19 07:04 PM, Monique Monteiro wrote:
Hi Jamie,
could you please tell us how much memory does your application consume
with Lucene? I'm asking it because we are having memory consumption problems
with a 32GB index and 1.5GB od RAM allocated to our web application. At the
momento, we use textual search.
Thanks in advance,
Monique
On Fri, Mar 19, 2010 at 8:49 AM, Jamie<ja...@stimulussoft.com> wrote:
Hi Guys
I just wanted to congratulate the Lucene guys for a fine job on 3.0!!
Since we switched our indexes to using integer based range queries based on
Date (YYMMHHSS), search speed is lightening fast and memory consumption has
dropped considerably!
Some stats:
Indexed Docs: 7.2M emails
Index Size: 24 GB (non optimized)
Search Speed: 0.06 - 0.09 seconds (with sort YYMMHHSS date)
Index stored on 4 SAS HDD hitachi RAID 10
16G RAM
2x Xeon 4 core 2.4Gz
OS FreeBSD 7.2
Filesystem UFS2 gjournal
I believe we are using all search performance recommendations now.
Good job!
Jamie
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
