As long as you have many document in index there can many unique terms in index. Every 128th term(by default) is written to term info index for faster term lookup. This info is loaded entirely to memory when searching so this can increase memory usage. Note that this does not depends on number of documents in index, it depends on number of unique terms in index.

This can be changed by setting higher value in indexWriter.setTermIndexInterval(); Be aware of setting this value too hight because search performance will degrade.
NOTE: this options is available only in Lucene 1.9.

Also it can depend on number of fields in document, the way you process them (store, index, tokenize etc.)


Leon Chaddock wrote:
Hi All,

We have a lucene index of over 10 000 000 docs at this time.
When we try and run a search we get
java.lang.OutOfMemoryError: Java heap space

We have tried setting the xmx settings to 1gb but to no avail (the box has 4gb of memory available) . IS there any guidance on handling memory or has anyone had similar problems before that could help?

Many thanks

Leon

----- Original Message ----- From: "Pradeep Sharma" <[EMAIL PROTECTED]>
To: <java-user@lucene.apache.org>
Sent: Wednesday, February 01, 2006 2:03 AM
Subject: Greetings and my first question - Is it a good practise to store application configuration in Lucene




I have just joined this user group, but I probably will be asking questions / contributing for a while now as I am starting to work on a product which will use Lucene exclusively.

Still in the designing phase, and I see that we need to manage several user / application specific configurations and I am exploring the idea of storing the configuration information also in the Index, may be create a separate index just for the configuration, because each module of the application will have access to Lucene classes.

I know technically this can be done, but are there any best practises which discourage this?

Thanks in advance.
-Pradeep



--------------------------------------------------------------------------------


No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.1.375 / Virus Database: 267.14.25/246 - Release Date: 30/01/2006


---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



--
regards,
Volodymyr Bychkoviak


---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Reply via email to