Harini Raghavan wrote:
I am using lucene 1.9.1 for search functionality in my j2ee application
using JBoss as app server. The lucene index directory size is almost 20G
right now. There is a Quartz job that is adding data to the index evey
min and around 20000 documents get added to the index every day.When the
documents are added and the segments are merged, the index size
increases and sometimes grows to more than double its original size.
This results in filling up the disk space. We have allotted a f/s size
of 50G and even that is not sufficient at times. Is there an optimum
vales for the f/s size to be allotted in such scenario.
Any suggestions would be appreciated.
I believe optimize should use at most 2X the starting index size,
transiently, if there are no readers open against the index. And then
when optimize is done the size should be around the starting size, or
less.
If there are open readers against the index when the optimize occurs,
then, the segments that were merged cannot actually be deleted until
those readers close. Even on Unix, where it will look like the
segments were deleted, they are still consuming disk space because
open file handles keep them allocated ("delete on last close").
This means if you have open readers you should see at most 3X the
starting index size. Worse, if some (but not all) readers are
re-opening while the merge is underway it's possible to peak at even
more than 3X the starting size.
Do you have readers running against your index?
I will call this out in the javadocs for optimize, addDocument,
addIndexes...
Mike
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
