Hi
Thanks for your reply. You right it looks as the original list is the
problem. The list I loop over is spring configured to return a list
of index searcher. Each index searcher looks at different indexes.
I would like to inject the list of index searchers as we may have
requirement to add new searchers. This would mean configuring spring
config file.
Trying to remove old index searcher gives me concurrent modification
exception. Hmmm.
Is there another approach I can take?
Cheers
Sent from my iPhone
On 21 Jan 2009, at 22:32, Erick Erickson <erickerick...@gmail.com>
wrote:
NOTE: you're iterating over 'searchers' and
adding to indexSearchers. Is that a typo?
Assuming that it's not and your 'searchers'
is the copy you talk about (so you can freely
add?) you never delete from the underlying
indexSearchers. But you do close elements
because you're closing a reference to the
searcher that points to the same underlying
object.
Assuming that's not the problem,
here's what I'd suggest. Log the count of
searchers just above your loop...
print searchers.size(); (or whatever).
for (IndexSearcher indexSearcher: searchers) {
Ian claims that the first time you'll see some number X
The second time you'll see X + Y.
You have to be getting this list of servers from someplace.
Wherever it is, it's (probably) persisted across calls,
because if it isn't you wouldn't have any open readers
to close.
Are you sure your local variable isn't just a reference
to the underlying (permanent) list?
See inline comments
for (IndexSearcher indexSearcher: searchers) {
IndexReader reader = indexSearcher.getIndexReader();
IndexReader newReader = reader.reopen();
if (newReader != reader) {
reader.close();
}
[EOE} you have not removed the instance of the searcher from
searchers (the
var in your for loop)
but you have closed it. So next time your code tries to
use it,
you've already closed it.
reader = newReader;
IndexSearcher indexSearch = new IndexSearcher(reader);
[EOE] This adds the newly opened searcher to the end of your array.
The
original (closed) one is still there.
indexSearchers.add(indexSearch);
}
[EOE] So if you use searchers anywhere from here on, it's got closed
readers in it if you closed any of them.
Best
Erick
On Wed, Jan 21, 2009 at 4:19 PM, Amin Mohammed-Coleman <ami...@gmail.com
>wrote:
Hi
I am trying to get an understanding and what the best practice is.
I am not saying that I am right, it may well be that my code is
wrong, that
is why I am posting this. The original loop that I am iterating
over is a
spring injected dependency. I don't reuse that in the
multisearcher. I
create a new list (local variable) when I invoke the search
method. So I'm
not sure how I can be adding to an existing list.
I presume it's a bad idea not to close the indexreader in this case.
Cheers
On 21 Jan 2009, at 20:43, Ian Lea wrote:
Oh well, it's your code so I guess you know what it does.
But I still think you're wrong.
If your list contains 3 searchers at the top of the loop and all 3
need to be reopened then the list will contain 6 searchers at the
end
of the loop, and the first 3 will be for readers that you've just
closed. Hence the already closed exception when you try to use
them.
--
Ian.
On Wed, Jan 21, 2009 at 8:24 PM, Amin Mohammed-Coleman <ami...@gmail.com
>
wrote:
Hi,
That is what I am doing with the line:
indexSearchers.add(indexSearch);
indexSearchers is an ArrayList that is constructed before the for
loop:
List<IndexSearcher> indexSearchers = new
ArrayList<IndexSearcher>();
I then pass the indexSearchers to :
multiSearcher = new MultiSearcher(indexSearchers.toArray(new
IndexSearcher[]
{}));
Cheers
On 21 Jan 2009, at 20:19, Ian Lea wrote:
I haven't been following this thread, but shouldn't you be
replacing
the old searcher in your list of searchers rather than just
adding the
new one on the end? Could be wrong - I find the names in your
code
snippet rather confusing.
--
Ian.
On Wed, Jan 21, 2009 at 6:59 PM, Amin Mohammed-Coleman <
ami...@gmail.com>
wrote:
Hi
I did the following according to java docs:
for (IndexSearcher indexSearcher: searchers) {
IndexReader reader = indexSearcher.getIndexReader();
IndexReader newReader = reader.reopen();
if (newReader != reader) {
reader.close();
}
reader = newReader;
IndexSearcher indexSearch = new IndexSearcher(reader);
indexSearchers.add(indexSearch);
}
First search works ok, susequent search result in:
org.apache.lucene.store.AlreadyClosedException: this
IndexReader is
closed
Cheers
On Wed, Jan 21, 2009 at 1:47 PM, Amin Mohammed-Coleman
<ami...@gmail.com>wrote:
Hi
Will give that a go.
Thanks
Sent from my iPhone
On 21 Jan 2009, at 12:26, "Ganesh" <emailg...@yahoo.co.in>
wrote:
I am closing the old reader and it is working fine for me.
Refer to
IndexReader.Reopen javadoc.
///Below is the code snipper from IndexReader.reopen javadoc
IndexReader reader = ...
...
IndexReader new = r.reopen();
if (new != reader) {
... // reader was reopened
reader.close(); //Old reader is closed.
}
reader = new;
Regards
Ganesh
----- Original Message ----- From: "Amin Mohammed-Coleman" <
ami...@gmail.com>
To: <java-user@lucene.apache.org>
Cc: <java-user@lucene.apache.org>
Sent: Wednesday, January 21, 2009 1:07 AM
Subject: Re: Indexing and Searching Web Application
Hi
Yes I am using the reopen method on indexreader. I am not
closing
the
old indexer as per Ganesh's instruction. It seems to be
working
correctly
so I presume it's ok not to close.
Thanks
Amin
On 20 Jan 2009, at 19:27, "Angel, Eric" <ean...@business.com>
wrote:
There's a reopen() method in the IndexReader class. You can
use
that.
-----Original Message-----
From: Amin Mohammed-Coleman [mailto:ami...@gmail.com]
Sent: Tuesday, January 20, 2009 5:02 AM
To: java-user@lucene.apache.org
Subject: Re: Indexing and Searching Web Application
Am I supposed to close the oldIndexReader? I just tried
this and I
get
an
exception stating that the IndexReader is closed.
Cheers
On Tue, Jan 20, 2009 at 9:33 AM, Ganesh <emailg...@yahoo.co.in
>
wrote:
Reopen the reader, only if it is modified.
IndexReader oldIndexReader = indexSearcher.getIndexReader();
if (!oldIndexReader.isCurrent()) {
IndexReader newIndexReader = oldIndexReader.reOpen();
oldIndexReader.close();
indexSearcher.close();
IndexSearcher indexSearch = new
IndexSearcher(newIndexReader);
}
Regards
Ganesh
----- Original Message ----- From: "Amin Mohammed-Coleman" <
ami...@gmail.com>
To: <java-user@lucene.apache.org>
Sent: Tuesday, January 20, 2009 1:38 PM
Subject: Re: Indexing and Searching Web Application
Hi
After your email I had a look around and came up with the
below
solution (I'm not sure if this is the right approach or
there is
a
performance implication to doing this)
public Summary[] search(SearchRequest searchRequest) {
List<Summary> summaryList = new ArrayList<Summary>();
StopWatch stopWatch = new StopWatch("searchStopWatch");
stopWatch.start();
MultiSearcher multiSearcher = null;
List<IndexSearcher> newIndexSearchers = new
ArrayList<IndexSearcher>();
try {
for (IndexSearcher indexSearcher: searchers) {
IndexReader indexReader =
indexSearcher.getIndexReader().reopen();
IndexSearcher indexSearch = new IndexSearcher(indexReader);
newIndexSearchers.add(indexSearch);
}
multiSearcher = new
MultiSearcher(newIndexSearchers.toArray(new
IndexSearcher[] {}));
QueryParser queryParser = new
MultiFieldQueryParser(
FieldNameEnum.fieldNameDescriptions(),
analyzer);
Query query =
queryParser.parse(searchRequest.getSearchTerm());
//TODO: Sort and Filters
TopDocs topDocs = multiSearcher.search(query, 100);
ScoreDoc[] scoreDocs = topDocs.scoreDocs;
LOGGER.debug("total number of hits for [" +
query.toString() + "
]
= " +topDocs.totalHits);
for (ScoreDoc scoreDoc : scoreDocs) {
final Document doc = multiSearcher.doc(scoreDoc.doc);
float score = scoreDoc.score;
final BaseDocument baseDocument = new BaseDocument(doc,
score);
Summary documentSummary = new
DocumentSummaryImpl(baseDocument);
summaryList.add(documentSummary);
}
} catch (Exception e) {
throw new IllegalStateException(e);
}
stopWatch.stop();
LOGGER.debug("total time taken for seach: " +
stopWatch.getTotalTimeMillis() + " ms");
return summaryList.toArray(new Summary[] {});
}
The searchers are configured in spring using which looks
like
this:
<bean id="rtfIndexSearcher"
class="org.apache.lucene.search.IndexSearcher"
scope="prototype"
lazy-
init="true" >
<constructor-arg type="org.apache.lucene.store.Directory"
ref="rtfDirectory" />
</bean>
I set the dependencies on the DocumentSearcher class.
Cheers
Amin
On 19 Jan 2009, at 21:45, Amin Mohammed-Coleman wrote:
I make a call to my search class which looks like this:
public Summary[] search(SearchRequest searchRequest) {
List<Summary> summaryList = new ArrayList<Summary>();
StopWatch stopWatch = new StopWatch("searchStopWatch");
stopWatch.start();
MultiSearcher multiSearcher = null;
try {
multiSearcher = new MultiSearcher(searchers.toArray(new
IndexSearcher[] {}));
QueryParser queryParser = new
MultiFieldQueryParser(
FieldNameEnum.fieldNameDescriptions(),
analyzer);
Query query =
queryParser.parse(searchRequest.getSearchTerm());
//TODO: Sort and Filters
TopDocs topDocs = multiSearcher.search(query, 100);
ScoreDoc[] scoreDocs = topDocs.scoreDocs;
LOGGER.debug("total number of hits for [" +
query.toString() +
" ] = " +topDocs.totalHits);
for (ScoreDoc scoreDoc : scoreDocs) {
final Document doc = multiSearcher.doc(scoreDoc.doc);
float score = scoreDoc.score;
final BaseDocument baseDocument = new BaseDocument(doc,
score);
Summary documentSummary = new
DocumentSummaryImpl(baseDocument);
summaryList.add(documentSummary);
}
} catch (Exception e) {
throw new IllegalStateException(e);
}
stopWatch.stop();
LOGGER.debug("total time taken for seach: " +
stopWatch.getTotalTimeMillis() + " ms");
return summaryList.toArray(new Summary[] {});
}
Do I need to do this explicitly?
Cheers
Amin
On 19 Jan 2009, at 20:48, Greg Shackles wrote:
After you make the commit to the index, are you
reloading the
index
in the
searchers?
- Greg
On Mon, Jan 19, 2009 at 3:29 PM, Amin Mohammed-Coleman <
ami...@gmail.com
wrote:
Hi
I have recently worked on developing an application
which
allows
you to
upload a file (which is indexed so you can search
later). I
have
numerous
tests to show that you can index and search documents
(in some
instances
within the same test), however when I perform the
operation in
the
site:
1) Upload File and Index
2) Search
I don't get any hits. When I restart the application
then if
I
make
another search I can find the results. It seems as though
indexes
aren't
being committed when I do the initial upload. This is
strange.
I
explicitly call commit in my code when I upload the
file. Has
anyone
experienced this before?
Any help would be appreciated.
Kind Regards
Amin
---
---
---
---
---
------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail:
java-user-h...@lucene.apache.org
Send instant messages to your online friends
http://in.messenger.yahoo.com
---
---
---
------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
---
---
---
------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
---
---
---
------------------------------------------------------------
To unsubscribe, e-mail: java-user-
unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
Send instant messages to your online friends
http://in.messenger.yahoo.com
---
---
---------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
---
------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
---
------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
