We have dual cpu intel xeon machines running "Red Hat Enterprise Linux
ES release 3 (Taroon Update 6)". We have 4GB memory on these machines
with 2GB allocated to tomcat.
After modifying the index we open a new one, warm it up, make it live
and then close the old one.
-siraj
Michael McCandless wrote:
A merge shouldn't make the machine completely non-responsive, just,
slower to run searches / index documents.
What kind of machine / IO system is this?
You can set maxMergeDocs to limit how large the merge is allowed to
be. But, be careful, since if you set this too small you'll wind up
with way too many segments over time, which'll slow down search and
risk file handle exhaustion. Likewise, increase mergeFactor. You
could also try decreasing the merge thread priority in
ConcurrentMergeScheduler... though that risk starvation of the merge
thread if your CPUs are really saturated doing indexing/searching.
Another thing to try is the BalancedSegmentMergePolicy (in
contrib/misc); it also tries to avoid big merges.
Also, how are you opening new readers? Can you share more how you are
using Lucene?
Mike
On Tue, Dec 22, 2009 at 5:57 PM, Siraj Haider <si...@jobdiva.com> wrote:
Hi Mike,
You are right, sometimes there is an implicit merge running when the machine
goes non-responsive. How can we avoid running those merges during the day
and how can we minimize the effect it will have on searches?
-siraj
Michael McCandless wrote:
Is it possible a large merge is running?
You can turn on IndexWriter.setInfoStream to see more details about
what IW is doing, including merging.
Mike
On Tue, Dec 22, 2009 at 5:19 PM, Siraj Haider <si...@jobdiva.com> wrote:
Hello guys,
We have a dilemma on a few of our lucene machines. We have a tomcat
running
our servlets for searching and indexing on each of these machines. Its a
live index where documents are being added to index while online searches
are also being served at the same time. Indexing happens every 5 minutes
and if there are new documents added, the index gets reopend. For most
of
the times the performance is very good, but under heavy load of searches,
the machine goes non-responsive. We can still telnet to machine and see
that cpu-wise its not bad, but I/O seems to be a problem. Is there
anything
we might be doing to cause it or anything that we can do to avoid it. I
know I did not provide a lot of information about how we are indexing and
searching but I will answer any question anyone might have.
thanks in advance
-siraj
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
