Re: IndexSearcher and IndexWriter.rollback

2022-04-15 Thread Gautam Worah
Q1. Are you using a SearcherManager or a direct IndexSearcher? If you are using a SearcherManager, you could just call `maybeRefresh()` and then re-acquire a new `IndexSearcher`. The method docs <https://lucene.apache.org/core/9_0_0/core/org/apache/lucene/search/ReferenceManager.html#maybeRefr

IndexSearcher and IndexWriter.rollback

2022-04-14 Thread erel
I’m using an IndexSearcher created from an IndexWriter (NRT mode). Up until now the IW was kept open forever. I want to properly handle cases where an indexing task failed and call IW.rollback to discard the incomplete changes. The problem I’m facing is that rollback also closes the writer

Re: Can an indexreader/indexsearcher survive index edits?

2021-09-22 Thread Gautam Worah
n rebuilding the entire index. > > I am also changing the search method so that I have a single service which > creates an IndexReader and IndexSearcher at startup, and reads and responds > to search requests through a socket. > > > > I know that an existing index can be maintain

Can an indexreader/indexsearcher survive index edits?

2021-09-22 Thread Trevor Nicholls
am changing this so that after the initial build subsequent data changes will update the index rather than rebuilding the entire index. I am also changing the search method so that I have a single service which creates an IndexReader and IndexSearcher at startup, and reads and responds to search

Re: Query on searchAfter API usage in IndexSearcher

2018-05-09 Thread Bryan Bende
Are you specifying a sort clause on your query? I'm not totally sure, but I think having a sort clause might be a requirement for efficient deep paging. I know Solr's cursorMark feature uses the searchAfter API, and a cursorMark is essentially the sort values of the last document from the previou

Re: Query on searchAfter API usage in IndexSearcher

2018-05-09 Thread Jacky Li
I have encountered the same problem, I wonder if anyone know the solution? Regards, Jacky -- Sent from: http://lucene.472066.n3.nabble.com/Lucene-Java-Users-f532864.html - To unsubscribe, e-mail: java-user-unsubscr...@lucene.a

Re: Query on searchAfter API usage in IndexSearcher

2018-05-09 Thread tomanishgupt...@gmail.com
Hi Lucene Team, Can you please reply to my query. Its a urgent issue and we need to resolve it at the earliest. Lucene Version used is 6.3.0 but even tried with the latest version 7.3.0. Regards Manish Gupta -- Sent from: http://lucene.472066.n3.nabble.com/Lucene-Java-Users-f532864.html

Query on searchAfter API usage in IndexSearcher

2018-05-07 Thread manish gupta
Hi Team, I am new to Lucene and I am trying to use Lucene for text search in my project to achieve better results in terms of query performance. Initially I was facing lot of GC issues while using lucene as I was using search API and passing all the documents count. As my data size is around 4 b

Re: Implement an IndexSearcher which never returns any documents

2017-09-13 Thread Mitchell Stevenson
thx, great solution 2017-09-11 17:55 GMT+02:00 Adrien Grand : > You could create a `new IndexSearcher(new MultiReader());` > > Le sam. 9 sept. 2017 à 19:40, Mitchell Stevenson < > mitchell.stevenson...@gmail.com> a écrit : > >> I need to implement an IndexSearcher for L

Re: Implement an IndexSearcher which never returns any documents

2017-09-11 Thread Adrien Grand
You could create a `new IndexSearcher(new MultiReader());` Le sam. 9 sept. 2017 à 19:40, Mitchell Stevenson < mitchell.stevenson...@gmail.com> a écrit : > I need to implement an IndexSearcher for Lucene 7 which never returns > any documents. > Is the following implementation su

Implement an IndexSearcher which never returns any documents

2017-09-09 Thread Mitchell Stevenson
I need to implement an IndexSearcher for Lucene 7 which never returns any documents. Is the following implementation suitable for this? The code seems to work nicely but i am not sure about it. IndexSearcher noDocsSearcher = new IndexSearcher(new NoDocsReader()); public class NoDocsReader

RE: Lucene IndexSearcher PrefixQuery seach getting really slow after a while

2016-11-04 Thread Nir Barel
Try to optimize your indexes. Sent securely from my iPhone From: Jason Wu Sent: Thursday, 3 November 2016 at 22:21:55 To: java-user@lucene.apache.org Subject: Lucene IndexSearcher PrefixQuery seach getting really slow after a while Hi Team, We are using lucene 4.8.1 to do some info searches

Lucene IndexSearcher PrefixQuery seach getting really slow after a while

2016-11-03 Thread Jason Wu
Hi Team, We are using lucene 4.8.1 to do some info searches every day for years. However, recently we encounter some performance issues which greatly slow down the lucene search. After application running for a while, we are facing below issues, which IndexSearcher PrefixQuery taking much

Re: Sorting IndexSearcher results by LongPoint with 6.0

2016-05-27 Thread Uwe Schindler
Hi Jeremy, Yes. That's right. The question is if you really need the stored field, but that's out of scope for this issue. Uwe Am 27. Mai 2016 01:21:48 MESZ, schrieb Jeremy Friesen : >Thanks for the help. So just to sum up, if I have a numeric field type >that >I want to be able to do a range q

Re: Sorting IndexSearcher results by LongPoint with 6.0

2016-05-26 Thread Jeremy Friesen
Thanks for the help. So just to sum up, if I have a numeric field type that I want to be able to do a range query on, sort by, and also retrieve in the document as a stored value, I will need to add it to the document three times, as a NumericDocValuesField, as a LongPoint, and as a StoredField. Do

Re: Sorting IndexSearcher results by LongPoint with 6.0

2016-05-26 Thread Uwe Schindler
Hi, Sorting does not work on indexed fields anymore (since Lucene 5), unless you use UninvertingReader. Point values don't work with that because they cannot be uninverted. For sorting it's the same rule for all field types: enable DocValues! You just have to add another field instance with sa

Sorting IndexSearcher results by LongPoint with 6.0

2016-05-26 Thread Jeremy Friesen
I'm attempting to upgrade my project to Lucene 6.0, and have run into an issue with sorting my results. My documents have a timestamp field that was previously a StoredField with NumericType: Long. I've converted it to a LongPoint, which seems to work fine for range queries. My problem is that try

Re: IndexReader returns all fields, but IndexSearcher does not

2015-06-02 Thread Ian Lea
ry an index. > When I try to read the index using IndexReader, I am able to print all the > fields (close to 30 fields stored) in the index. > However, when I run a query on the same index using IndexSearcher, I am > able to get only a couple of fields instead of all the fields as re

IndexReader returns all fields, but IndexSearcher does not

2015-06-01 Thread Rahul Kotecha
Hi All, I am trying to query an index. When I try to read the index using IndexReader, I am able to print all the fields (close to 30 fields stored) in the index. However, when I run a query on the same index using IndexSearcher, I am able to get only a couple of fields instead of all the

Re: IndexSearcher creation policy question

2014-08-22 Thread Sheng
can be updated at any time by users. > > What I understand is that: > > - I need a fresh IndexSearcher for each initial query (DirectoryReader.open > -> reader -> searcher) and cannot use Search(Lifetime)Manager's. > - I cannot share IndexSearchers in the depicted scenar

IndexSearcher creation policy question

2014-08-22 Thread Rolf Veen
dated at any time by users. What I understand is that: - I need a fresh IndexSearcher for each initial query (DirectoryReader.open -> reader -> searcher) and cannot use Search(Lifetime)Manager's. - I cannot share IndexSearchers in the depicted scenario; even for the same user, a differe

Re: absence of searchAfter method with Collector parameter in Lucene IndexSearcher

2014-06-08 Thread Kailash Budhathoki
quot; ScoreDoc as per the documentation. But this is somehow not working. I would appreciate your suggestions. Best, -- Kailash Budhathoki On Sat, Jun 7, 2014 at 12:00 AM, Chris Hostetter wrote: > > : I was wondering why there is no search method in lucene Indexsearcher to > : searc

Re: absence of searchAfter method with Collector parameter in Lucene IndexSearcher

2014-06-06 Thread Chris Hostetter
: I was wondering why there is no search method in lucene Indexsearcher to : search after last reference by passing collector. Say a method with : signature like searchAfter(Query query, ScoreDoc after, Collector results). searchAfter only makes sense if there is a Sort involved -- either

absence of searchAfter method with Collector parameter in Lucene IndexSearcher

2014-06-06 Thread Kailash Budhathoki
Hi, I was wondering why there is no search method in lucene Indexsearcher to search after last reference by passing collector. Say a method with signature like searchAfter(Query query, ScoreDoc after, Collector results). For normal search there are two ways to search; one by passing the collector

Re: How to make good use of the multithreaded IndexSearcher?

2013-10-01 Thread Benson Margulies
to control the > parallelism and then forceMerge each shard down to 1 segment to avoid > spawning extra threads per shard. Once that's done, you just open all of > the shards with a MultiReader and use that with the IndexSearcher and an > ExecutorService. > > The downside to this is

Re: How to make good use of the multithreaded IndexSearcher?

2013-10-01 Thread Desidero
avoid spawning extra threads per shard. Once that's done, you just open all of the shards with a MultiReader and use that with the IndexSearcher and an ExecutorService. The downside to this is that it doesn't play nicely with near real-time search, but if you have a relatively static index

Re: How to make good use of the multithreaded IndexSearcher?

2013-10-01 Thread Michael McCandless
ere are alwas #cores queries running in parrallel), > there is generally no need to use the multi-threaded IndexSearcher. > The multi-threaded index searcher can however help in case all CPU > power is not in use or if you care more about latency than throughput. > It indeed leverages

Re: How to make good use of the multithreaded IndexSearcher?

2013-10-01 Thread Adrien Grand
the multi-threaded IndexSearcher. The multi-threaded index searcher can however help in case all CPU power is not in use or if you care more about latency than throughput. It indeed leverages the fact that the index is splitted into segments to parallelize query execution, so a fully merged index wi

How to make good use of the multithreaded IndexSearcher?

2013-09-30 Thread Benson Margulies
The multithreaded index searcher fans out across segments. How aggressively does 'optimize' reduce the number of segments? If the segment count goes way down, is there some other way to exploit multiple cores?

RE: IndexSearcher using Collector

2013-06-19 Thread Uwe Schindler
nal Message- > From: amit nanda [mailto:amit...@gmail.com] > Sent: Wednesday, June 19, 2013 10:59 AM > To: java-user@lucene.apache.org > Subject: IndexSearcher using Collector > > Hi, > > I have multiple index that i want to search against, thus i am using > MultiReader fo

IndexSearcher using Collector

2013-06-19 Thread amit nanda
Hi, I have multiple index that i want to search against, thus i am using MultiReader for that. Along with this I also want all the matches to the query so i am using Collector class for this. The issue i am facing is that I am not able to know when all the matches are done, i.e. for each matching

Re: Necessary to close() IndexSearcher in 4.X?

2013-04-03 Thread Lewis John Mcgibbney
Thanks for feeback Uwe. I'll not be looking at this until again tomorrow so at least this gives me time to think it through. On Wednesday, April 3, 2013, Uwe Schindler wrote: > Hi, > > In Lucene before 4.0 there was a close method in IndexSearcher, because you were able to create

RE: Necessary to close() IndexSearcher in 4.X?

2013-04-03 Thread Uwe Schindler
Hi, In Lucene before 4.0 there was a close method in IndexSearcher, because you were able to create IndexSearcher using Directory, which internally opened an IndexReader. This IndexReader had to be closed, so there was a need for IndexSearcher.close(). In 3.x this was constructor (taking

Necessary to close() IndexSearcher in 4.X?

2013-04-03 Thread Lewis John Mcgibbney
Hi, I am encountering many situations where searcher.close() is present in finally blocks such as } finally { if (searcher != null) { try { searcher.close(); } catch (Exception ignore) { } searc

Re: How to get field names and types from an IndexSearcher

2013-02-06 Thread Rolf Veen
Just for the record, the solution that I adopted is as follows: - Create a setType(String field, String type) and call it for any known numeric fields, before adding any document. This method saves the type definition in a file and also sets the Map that is used in the method StandardQueryPa

Re: How to get field names and types from an IndexSearcher

2013-02-01 Thread Rolf Veen
On Fri, Feb 1, 2013 at 12:43 PM, Michael McCandless wrote: > There is actually one way to check if a field was indexed numerically: > you can seek to the first term in the field, and attempt to parse it > as a long/float/etc., and if that throws a NumberFormatException, it > was indexed numerical

Re: How to get field names and types from an IndexSearcher

2013-02-01 Thread Michael McCandless
On Fri, Feb 1, 2013 at 3:17 AM, Rolf Veen wrote: > On Thu, Jan 31, 2013 at 9:55 PM, Michael McCandless > wrote: > >> But are you wanting to, eg, make a NumericRangeQuery if you detect the >> field was indexed numerically, and otherwise a TermRangeQuery, or >> something...? (Not easy) > > This is

Re: How to get field names and types from an IndexSearcher

2013-02-01 Thread Rolf Veen
On Thu, Jan 31, 2013 at 9:55 PM, Michael McCandless wrote: > But are you wanting to, eg, make a NumericRangeQuery if you detect the > field was indexed numerically, and otherwise a TermRangeQuery, or > something...? (Not easy) This is what I want, yes. But I begin to understand that this is not

Re: How to get field names and types from an IndexSearcher

2013-01-31 Thread Michael McCandless
On Thu, Jan 31, 2013 at 7:31 AM, Rolf Veen wrote: > Thank you, Mike. > > I didn't state why I need this. I want to be able to send > a query to some QueryParser that understands "field:1" > regardless if 'field' was added as StringField or LongField, > for example. I do not want to rely on schema

Re: How to get field names and types from an IndexSearcher

2013-01-31 Thread Rolf Veen
Field class was used for the > indexing: that information is not fully preserved ... > > Mike McCandless > > http://blog.mikemccandless.com > > On Thu, Jan 31, 2013 at 6:33 AM, Rolf Veen wrote: >> Hello, all. >> >> I want to get a list of field names and types out

Re: How to get field names and types from an IndexSearcher

2013-01-31 Thread Michael McCandless
wrote: > Hello, all. > > I want to get a list of field names and types out of an IndexSearcher > or IndexReader (not necesarily Atomic). By type I mean if it was > stored as StringField, LongField, etc. Is this possible ? > > I could get the field names this way, probably not t

RE: How to properly refresh MultiReader IndexSearcher in Lucene 4.0-BETA

2012-08-28 Thread Uwe Schindler
er to keep the alive, this is much easier than doing it yourself). Once you need to reopen one, just reopen it and save it. - On *every* search create a new MultiReader() [this costs nothing, as it is just a wrapper] and wrap it with a new IndexSearcher [this also costs you nothing, as it is a

How to properly refresh MultiReader IndexSearcher in Lucene 4.0-BETA

2012-08-27 Thread Mossaab Bagdouri
me up with this solution. I just want to share, and get feedbacks if any. The idea is to create a new instance of MultiReader, add the old current SubReaders and the new changed ones, refresh the IndexSearcher, then close the old out-of-date SubReaders. private IndexSearcher getIndexSearcher() {

Re: Hanging with fixed thread pool in the IndexSearcher multithread code

2012-02-20 Thread Benson Margulies
On Sun, Feb 19, 2012 at 10:39 PM, Trejkaz wrote: > On Mon, Feb 20, 2012 at 12:07 PM, Uwe Schindler wrote: >> See my response. The problem is not in Lucene; its in general a problem of >> fixed >> thread pools that execute other callables from within a callable running at >> the >> moment in the

Re: Hanging with fixed thread pool in the IndexSearcher multithread code

2012-02-19 Thread Trejkaz
On Mon, Feb 20, 2012 at 12:07 PM, Uwe Schindler wrote: > See my response. The problem is not in Lucene; its in general a problem of > fixed > thread pools that execute other callables from within a callable running at > the > moment in the same thread pool. Callables are simply waiting for each

Re: Hanging with fixed thread pool in the IndexSearcher multithread code

2012-02-19 Thread Benson Margulies
java-user@lucene.apache.org >> Subject: Re: Hanging with fixed thread pool in the IndexSearcher multithread >> code >> >> See https://issues.apache.org/jira/browse/LUCENE-3803 for an example of the >> hang. I think t

RE: Hanging with fixed thread pool in the IndexSearcher multithread code

2012-02-19 Thread Uwe Schindler
2 1:47 AM > To: java-user@lucene.apache.org > Subject: Re: Hanging with fixed thread pool in the IndexSearcher multithread > code > > See https://issues.apache.org/jira/browse/LUCENE-3803 for an example of the > hang. I think this nets out to pilot error, but maybe Javadoc could p

Re: Hanging with fixed thread pool in the IndexSearcher multithread code

2012-02-19 Thread Benson Margulies
See https://issues.apache.org/jira/browse/LUCENE-3803 for an example of the hang. I think this nets out to pilot error, but maybe Javadoc could protect the next person from making the same mistake. - To unsubscribe, e-mail: java-u

Re: Hanging with fixed thread pool in the IndexSearcher multithread code

2012-02-19 Thread Benson Margulies
e with one thread, and >> then with two threads, to the IndexSearcher constructor. >> >> It hung. >> >> With three threads, it didn't work, but I got different results than >> when I don't pass in an executor service at all. >> >> Is this expec

Re: Hanging with fixed thread pool in the IndexSearcher multithread code

2012-02-19 Thread Benson Margulies
wo threads, to the IndexSearcher constructor. >> >> It hung. >> >> With three threads, it didn't work, but I got different results than >> when I don't pass in an executor service at all. >> >> Is this expected? Should the javadoc say something? (

Re: Hanging with fixed thread pool in the IndexSearcher multithread code

2012-02-19 Thread Benson Margulies
19, 2012 at 9:08 AM, Benson Margulies > wrote: >> 3.5.0: I passed a fixed size executor service with one thread, and >> then with two threads, to the IndexSearcher constructor. >> >> It hung. >> >> With three threads, it didn't work, but I got different

Re: Hanging with fixed thread pool in the IndexSearcher multithread code

2012-02-19 Thread Robert Muir
On Sun, Feb 19, 2012 at 9:08 AM, Benson Margulies wrote: > 3.5.0:  I passed a fixed size executor service with one thread, and > then with two threads, to the IndexSearcher constructor. > > It hung. > > With three threads, it didn't work, but I got different results than &g

Hanging with fixed thread pool in the IndexSearcher multithread code

2012-02-19 Thread Benson Margulies
3.5.0: I passed a fixed size executor service with one thread, and then with two threads, to the IndexSearcher constructor. It hung. With three threads, it didn't work, but I got different results than when I don't pass in an executor service at all. Is this expected? Should the j

Re: Filter and IndexSearcher in Lucene 4.0 (trunk)

2012-02-10 Thread Hany Azzam
gt; Sent: Friday, February 10, 2012 6:43 PM >> To: java-user@lucene.apache.org >> Subject: Re: Filter and IndexSearcher in Lucene 4.0 (trunk) >> >> Hi, >> >> I apologise upfront for the trivial question. I have an IndexSearcher and > I am >> applying a FieldCacheTe

RE: Filter and IndexSearcher in Lucene 4.0 (trunk)

2012-02-10 Thread Uwe Schindler
gt; Subject: Re: Filter and IndexSearcher in Lucene 4.0 (trunk) > > Hi, > > I apologise upfront for the trivial question. I have an IndexSearcher and I am > applying a FieldCacheTermsFilter filter on it to only retrieve documents whose > single docId is in a provided set of

Re: Filter and IndexSearcher in Lucene 4.0 (trunk)

2012-02-10 Thread Hany Azzam
Hi, I apologise upfront for the trivial question. I have an IndexSearcher and I am applying a FieldCacheTermsFilter filter on it to only retrieve documents whose single docId is in a provided set of allowed docIds. I am particularly interested in the stats being estimated over the accepted set

Re: IndexSearcher with two Indexes

2012-01-27 Thread Robert Muir
IndexReader documents = IndexReader.open(documentsDirectory); final IndexSearcher relevantSearcher = new IndexSearcher(relevant); IndexSearcher documentsSearcher = new IndexSearcher(documents) { @Override public CollectionStatistics collectionStatistics(String field) thro

Re: IndexSearcher with two Indexes

2012-01-27 Thread Hany Azzam
tions). One of the retrieval models requires statistics to be computed >> across both of the indexes. How can an IndexSearcher use the two indexes at >> the same time to compute different components of the retrieval model? Is >> that possible? >> > > you can make a mul

Re: IndexSearcher with two Indexes

2012-01-27 Thread Robert Muir
arity functions). > One of the retrieval models requires statistics to be computed across both of > the indexes. How can an IndexSearcher use the two indexes at the same time to > compute different components of the retrieval model? Is that possible? > you can make a multireader over

Re: IndexSearcher with two Indexes

2012-01-27 Thread Hany Azzam
computed across both of the indexes. How can an IndexSearcher use the two indexes at the same time to compute different components of the retrieval model? Is that possible? Thank you very much, Hany

Re: RE: Question about FilterIndexReader and IndexSearcher

2011-06-27 Thread 周洲
Hi, I'am a student of Southeast University which locate in China, thank you for your help,but i still cann't filter the docs being deleted,i make a test demo,please tell me why the following procedure will be such a result? Why would IndexSearcher ignore the deleted docs

Question about FilterIndexReader and IndexSearcher

2011-06-26 Thread 周洲
Hello, I want to let IndexReader finding the modification in time,so i use MyFilterIndexReader which extend FilterIndexReader to cache the deleted document in RAM.when this FilterIndexReader be the argument of a IndexSearcher,i found that this IndexSearcher can not filter the deleted document

RE: Question about FilterIndexReader and IndexSearcher

2011-06-26 Thread Uwe Schindler
Hi, usage of FilterIndexReader is not always as easy as it seems. There are several problem, that can easy lead to the fact that you FilterIndexReader implements all document filtering, but IndexSearcher does not respect it. I have no idea what you are doing, but the following thing need to be

Question about FilterIndexReader and IndexSearcher

2011-06-25 Thread 周洲
Hello, I want to let IndexReader finding the modification in time,so i use MyFilterIndexReader which extend FilterIndexReader to cache the deleted document in RAM.when this FilterIndexReader be the argument of a IndexSearcher,i found that this IndexSearcher can not filter the deleted document

Re: Lucene: Indexsearcher: java.lang.UnsupportedOperationException

2011-04-19 Thread Patrick Diviacco
java.lang.UnsupportedOperationException at org.apache.lucene.search.Query.createWeight(Query.java:88) at org.apache.lucene.search.BooleanQuery$BooleanWeight.(BooleanQuery.java:185) at org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:360) at org.apache.lucene.search.Query.weight(

Re: Lucene: Indexsearcher: java.lang.UnsupportedOperationException

2011-04-19 Thread Anshum
Could you also print and send the entire stack-trace? Also, the query.toString() -- Anshum Gupta http://ai-cafe.blogspot.com On Tue, Apr 19, 2011 at 7:40 PM, Patrick Diviacco < patrick.divia...@gmail.com> wrote: > I get the following error message: java.lang.UnsupportedOperationException > > wi

Lucene: Indexsearcher: java.lang.UnsupportedOperationException

2011-04-19 Thread Patrick Diviacco
I get the following error message: java.lang.UnsupportedOperationException with Lucene search method: topDocs = searcher.search(booleanQuery, null, 100); I'm using an old version of Lucene: Lucene 2.4.1 (I cannot upgrade!) Can you help me to understand why I get such error ? thanks This is the c

Re: IndexSearcher Single Instance Bottleneck?

2011-03-10 Thread Erick Erickson
iew this message in context: > http://lucene.472066.n3.nabble.com/IndexSearcher-Single-Instance-Bottleneck-tp2662376p2662376.html > Sent from the Lucene - Java Users mailing list archive at Nabble.com. > > - > To unsubs

IndexSearcher Single Instance Bottleneck?

2011-03-10 Thread RobM
/IndexSearcher-Single-Instance-Bottleneck-tp2662376p2662376.html Sent from the Lucene - Java Users mailing list archive at Nabble.com. - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail

Re: shared IndexSearcher (lucene 3.0.3)

2011-02-25 Thread Simon Willnauer
come back here! Simon On Fri, Feb 25, 2011 at 2:11 PM, Akos Tajti wrote: > Hi all, > > in our project we're using lucene in tomcat. To avoid some overhead we have > a shared IndexSearcher instance. In the past we had too many open files > errors many times. To prevent this

shared IndexSearcher (lucene 3.0.3)

2011-02-25 Thread Akos Tajti
Hi all, in our project we're using lucene in tomcat. To avoid some overhead we have a shared IndexSearcher instance. In the past we had too many open files errors many times. To prevent this the IndexSearcher is closed and reopened after indexing. The shared instance is not closed anywhere

Re: Newbie: "Life span" of IndexWriter / IndexSearcher?

2011-01-16 Thread sol myr
Worked like a charm - thanks a lot. --- On Sun, 1/16/11, Raf wrote: From: Raf Subject: Re: Newbie: "Life span" of IndexWriter / IndexSearcher? To: java-user@lucene.apache.org Date: Sunday, January 16, 2011, 3:16 AM Look at the JavaDoc: http://lucene.apache.org/java/3_0_2/api/core/

Re: Newbie: "Life span" of IndexWriter / IndexSearcher?

2011-01-16 Thread Raf
= reader.reopen(true); if (newReader != reader) { reader.close(); reader = newReader; searcher = new IndexSearcher(reader); } instead of reader.reopen(true); Bye. *Raf* On Sun, Jan 16, 2011 at 11:06 AM, sol myr wrote: > Hi, > > Thank you kindly for replying. > Unfortunately, reopen() doesn&#

RE: Newbie: "Life span" of IndexWriter / IndexSearcher?

2011-01-16 Thread sol myr
t(makeDoc("hello world 1")); // Field title="hello world 1" writer.commit(); // First search (yields document #1 as expected): IndexReader reader=IndexReader.open(directory, true); IndexSearcher searcher = new IndexSearcher(reader); TopDocs results1 = searcher.search(query, 1); print

Re: Closing indexsearcher , making sur eit is in use

2011-01-13 Thread Umesh Prasad
You can use ReadWriteLock as low level technique to manage access. A ReadWriteLock maintains a pair of associated locks

Re: Closing indexsearcher , making sur eit is in use

2011-01-13 Thread Ian Lea
Use something with reference counting - Lucene in action second edition has a searcher manager class which I think might be available standalone. Or a couple of low-tech alternatives: instead of closing the old searcher, move it out of the way and keep a reference to it and close it after n secon

Closing indexsearcher , making sur eit is in use

2011-01-13 Thread Paul Taylor
As recommended, I use just one Index Searcher on my multithreaded GUI app using a singleton pattern If data is modified in the index I then close the reader and searcher, and they will be recreate on next call to getInstance() but Ive hit a problem whereby one thread was closing a searcher, anot

RE: Newbie: "Life span" of IndexWriter / IndexSearcher?

2011-01-13 Thread Uwe Schindler
Hi, > We're writing a web application, which naturally needs > - "IndexSearcher" when users use our search screen > - "IndexWriter" in a background process that periodically updates and > optimizes our index. > Note our writer is exclusive - no othe

Newbie: "Life span" of IndexWriter / IndexSearcher?

2011-01-13 Thread sol myr
Hi, We're writing a web application, which naturally needs - "IndexSearcher" when users use our search screen - "IndexWriter" in a background process that periodically updates and optimizes our index. Note our writer is exclusive - no other applications/threads ever

Re: Can not delete index file after close the IndexSearcher

2011-01-13 Thread Ian Lea
In fact it's probably as simple as if (searcher == null) { searcher = new IndexSearcher(dir); } at the top of the loop. -- Ian. 2011/1/13 Ian Lea : > As I said, there is probably a better solution. At the moment you are > opening searchers at the top and bottom of the

Re: Can not delete index file after close the IndexSearcher

2011-01-13 Thread 张志田
u are not closing the bottom one, that you've only > just opened, before opening a new one using the same instance > variable. The resources of the bottom one would presumably be > released eventually by GC, but evidently not soon enough, > > Replace the top searcher = new

Re: Can not delete index file after close the IndexSearcher

2011-01-13 Thread Ian Lea
urces of the bottom one would presumably be released eventually by GC, but evidently not soon enough, Replace the top searcher = new IndexSearcher(dir); line with if (needToOpenNewSearcher()) { ... } where the logic in needToOpenNewSearcher() is for you to write. -- Ian. 2011/1/13 张志田 :

Re: Can not delete index file after close the IndexSearcher

2011-01-13 Thread 张志田
try { searcher.close(); } catch (Exception e) { } > > before > > searcher = new IndexSearcher(dir); > > at the top of the loop. > > At the end of a loop searcher is open, and is not closed before being > reassigned. There is probably a better solution along the lin

Re: Can not delete index file after close the IndexSearcher

2011-01-13 Thread Ian Lea
Try adding try { searcher.close(); } catch (Exception e) { } before searcher = new IndexSearcher(dir); at the top of the loop. At the end of a loop searcher is open, and is not closed before being reassigned. There is probably a better solution along the lines of only opening new

Re: Can not delete index file after close the IndexSearcher

2011-01-12 Thread 张志田
ssing the directory needs to be closed before opening a new one. > > dir.close(); > dir = FSDirectory.open(new File(getIndexPath())); > > why not to open two IndexSearcher objects in an array of two instead of > swapping them back and forth? > it would be a lot easier. > > yu

Re: Can not delete index file after close the IndexSearcher

2011-01-12 Thread Yuhan Zhang
Hi Garry, I am guessing the directory needs to be closed before opening a new one. dir.close(); dir = FSDirectory.open(new File(getIndexPath())); why not to open two IndexSearcher objects in an array of two instead of swapping them back and forth? it would be a lot easier. yuhan 2011/1/12 张志田

Re: Can not delete index file after close the IndexSearcher

2011-01-12 Thread 张志田
yzer = new StandardAnalyzer(Version.LUCENE_30); private IndexSearcher searcher; public void search() { while (true) { try { String keyword = "test"; String fieldName = "searchfield";

Re: Can not delete index file after close the IndexSearcher

2011-01-12 Thread Michael McCandless
int MAX_RESULT = 1; >> > >> >private String indexPath1 = "D:\\index\\index1"; >> >private String indexPath2 = "D:\\index\\index2"; >> > >> >private String indexPath = indexPath1; >> > >> >private Analyzer an

Re: Can not delete index file after close the IndexSearcher

2011-01-12 Thread 张志田
x2"; > > > >private String indexPath = indexPath1; > > > >private Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_30); > > > >private Directory dir = null; > > > >private IndexSearcher searcher; > > > >

Re: Can not delete index file after close the IndexSearcher

2011-01-12 Thread Michael McCandless
ss SearchTest > { > >    private static final int MAX_RESULT = 1; > >    private String indexPath1 = "D:\\index\\index1"; >    private String indexPath2 = "D:\\index\\index2"; > >    private String indexPath = indexPath1; > >    private Analyzer analyze

Can not delete index file after close the IndexSearcher

2011-01-12 Thread 张志田
dir = null; private IndexSearcher searcher; public void search() { while(true) { try { String keyword = "test"; String fieldName = "searchfield"; if(dir == null) {

Weird document equals and hash through IndexReader & IndexSearcher

2010-11-04 Thread Carmit Sahar
va-user@lucene.apache.org Subject: Weird document equals and hash through IndexReader & IndexSearcher Hi, I have a weird result: If I access the same document through the IndexReader or IndexSearcher, they are not equal and have different hash values: Docume

RE: Weird document equals and hash through IndexReader & IndexSearcher

2010-11-04 Thread Uwe Schindler
...@thetaphi.de > -Original Message- > From: Carmit Sahar [mailto:carmi...@gmail.com] > Sent: Thursday, November 04, 2010 9:27 AM > To: java-user@lucene.apache.org > Subject: Weird document equals and hash through IndexReader & > IndexSearcher > > Hi, >

Weird document equals and hash through IndexReader & IndexSearcher

2010-11-04 Thread Carmit Sahar
Hi, I have a weird result: If I access the same document through the IndexReader or IndexSearcher, they are not equal and have different hash values: Document doc1 = indexSearcher.doc(i); Document doc2 = indexSearcher.getIndexReader().document(i

Re: Does a IndexSearcher call incRef on the underlying reader?

2010-10-27 Thread Michael McCandless
On Wed, Oct 27, 2010 at 1:01 PM, Pulkit Singhal wrote: > 1st of all, great book. Thank you! > @Question3: It sounds like an IndexReader always starts with a count of zero > but that should not be a cause of worry because the value only gets acted > upon in a call to decRef() ... am I right? Ac

Re: Blocking on IndexSearcher search

2010-09-07 Thread Paul Taylor
Uwe Schindler wrote: Im using Windows and I'll try NIO, good idea, my app is already memory hungry in other areas so I guess MMapped is a no go, doe sit use heap or perm memory ? It uses address space for mapping the files into virtual memory (like a swap file) - this is why it o

RE: Blocking on IndexSearcher search

2010-08-25 Thread Uwe Schindler
> Im using Windows and I'll try NIO, good idea, my app is already memory > hungry in other areas so I guess MMapped is a no go, doe sit use heap or perm > memory ? It uses address space for mapping the files into virtual memory (like a swap file) - this is why it only works well for 64bit VMs. The

Re: Blocking on IndexSearcher search

2010-08-25 Thread Paul Taylor
Uwe Schindler wrote: That lock contention is fine there as this is the central point where all IO is done. This does not mean that only one query is running in parallel, the queries are still running in parallel. But there is one place where all IO is waiting for one file descriptor. This is not

Re: Blocking on IndexSearcher search

2010-08-25 Thread Paul Taylor
ndexWriter(directory,analyzer,true, IndexWriter.MaxFieldLength.UNLIMITED); and my Index Searcher is created as IndexSearcher is = new IndexSearcher(directory,true); Paul - To unsubscribe, e-mail: java-user-unsubscr...@lucene.

RE: Blocking on IndexSearcher search

2010-08-25 Thread Uwe Schindler
? - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Paul Taylor [mailto:paul_t...@fastmail.fm] > Sent: Wednesday, August 25, 2010 10:26 PM > To: java-user@lucene.apache.org > Subject: Blocking o

  1   2   3   4   >