have a look at LuceneQueryOptimizer.java in nutch
----- Original Message ----
From: Tim Johnson <[EMAIL PROTECTED]>
To: java-user@lucene.apache.org
Sent: Wednesday, 21 February, 2007 3:34:36 PM
Subject: Stop long running queries
I'm having issues with some queries taking in excess of 500 secs to run to
completion. The system being used consists of ~100 million docs spilt up
across ~600 indexes. The indexes are of various sizes from 15MB to 8GB and
all searches done in the system require an exact count of matching hits.
The documents are also indexed with various fields and types ranging from 15
to 200 fields depending on the data indexed.
98% of all queries run under 4 sec, but the remaining 2% run for > 15 secs
and sometimes > 500 secs. These long running queries usually consist of
wildcard and range queries where the number of BooleanQuery statements
reaches into the 100,000 range. Some of these queries return a relatively
small number of results (below 300).
I've recently tried to kill the search using a separate thread to wrap the
searcher.search method, which works, but only once. The next query to come
through throws an "IOException: The handle is invalid." in the
FSInputStream.readInternal() method.
Any suggestions on how to stop these long running queries?
Tim Johnson
SAIC
Systems Engineer
757.226.1569
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
___________________________________________________________
What kind of emailer are you? Find out today - get a free analysis of your
email personality. Take the quiz at the Yahoo! Mail Championship.
http://uk.rd.yahoo.com/evt=44106/*http://mail.yahoo.net/uk
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
