Re: Caching lucene index

We do not have anything that is designed to address this use-case, except maybe MemoryIndex if your use-case involves indexing a single document. I don't really have suggestions except making sure that the number of indices that you maintain at any point in time is bounded. Lucene indices typically

Re: Caching analyzed query

What kind of queries are these? I.e. How much work goes into step 4? Is this a fairly standard combination of Boolean/Phrase/other stock Lucene queries built up out of tokenizing the text? If so, it's going to be nowhere near the bottleneck in your runtime (we're talking often way less than a mi

RE: caching an indexreader

Thanks for the comments. Sounds like I will probably be ok. -Original Message- From: Jason Rutherglen [mailto:jason.rutherg...@gmail.com] Sent: Friday, June 19, 2009 1:50 PM To: java-user@lucene.apache.org; java-...@lucene.apache.org Subject: Re: caching an indexreader On the topic of

Re: caching an indexreader

On the topic of RAM consumption, it seems like field caches could return estimated RAM usage (given they're arrays of standard Java types)? There's methods of calculating per platform (I believe relatively accurately). On Fri, Jun 19, 2009 at 12:11 PM, Michael McCandless < luc...@mikemccandless.co

Re: caching an indexreader

On Fri, Jun 19, 2009 at 2:40 PM, Scott Smith wrote: > In my environment, one of the concerns is that new documents are > constantly being added (and some documents may be deleted).  This means > that when a user does a search and pages through results, it is possible > that there are new items comi

Re: caching an indexreader

> As I understand it, the user won't see any changes to the index until a new Searcher is created. Correct. > How much memory will caching the searcher cost? Are there other tradeoff's I need to consider? If you're updating the index frequently (every N seconds) and the searcher/reader is closed

Re: caching fields for query performance

Yes, pull out language:ENG language:FRA language:RUS into filters, cache them, and re-use them across all queries. In Lucene, see CachingWrapperFilter() In Solr, use a separate fq (for filter query) parameter... q=+topic:m&a +topic:earn +company:MSFT&fq=language:ENG -Yonik On Fri, Jul 25, 200

Re: Caching in lucene

18 sep 2007 kl. 08.53 skrev Shailendra Mudgal: Ok let me explain you. By warming up the reader i used to understand is that it memorizes the index terms. Therefore subsequent queries will be answered by using these terms. Is this correct ? Are you asking if all terms are flyweighted/inte

Re: Caching in lucene

Ok let me explain you. By warming up the reader i used to understand is that it memorizes the index terms. Therefore subsequent queries will be answered by using these terms. Is this correct ? On 9/18/07, Karl Wettin <[EMAIL PROTECTED]> wrote: > > > 18 sep 2007 kl. 08.33 skrev Shailendra Mudgal:

Re: Caching in lucene

18 sep 2007 kl. 08.33 skrev Shailendra Mudgal: Whether it caches frequently used terms ? I don't understand you question? -- karl On 9/18/07, Karl Wettin <[EMAIL PROTECTED]> wrote: 18 sep 2007 kl. 07.12 skrev Shailendra Mudgal: What my goal is to understand the caching strategy.

Re: Caching in lucene

Whether it caches frequently used terms ? On 9/18/07, Karl Wettin <[EMAIL PROTECTED]> wrote: > > > 18 sep 2007 kl. 07.12 skrev Shailendra Mudgal: > > > > > What my goal is to understand the caching strategy. How well this > > work for repetitive queries. Is there any room available to improve >

Re: Caching in lucene

18 sep 2007 kl. 07.12 skrev Shailendra Mudgal: What my goal is to understand the caching strategy. How well this work for repetitive queries. Is there any room available to improve this. It is usually a loss of resources to cache results in a busy system with gaussianity distributed q

Re: Caching in lucene

Hi Yonik, Thanks for your response. I'll feel great if you can explain this in more detail as i am not sure that whether i have understood this correctly or not. Or if you can direct me to some resource that will also be very good for me. What my goal is to understand the caching strategy. How w

Re: Caching in lucene

On 9/17/07, Shailendra Mudgal <[EMAIL PROTECTED]> wrote: > One thing that i understand about IndexReader is that for subsequent > queries, results come fast as the IndexReader needs to be warmed up. > According to this, I am trying to find out the answers of following > questions : > - is there any

Re: Caching IndexSearcher in a webapp [was: Find "latest" document (before a certain date)]

Hi, Answers in the text. > For each search request (it's a webapp) I currently create > a new IndexSearcher, new Filter and new Sort, call > searcher.search(query, filter, sorter) and later searcher.close(). > > The literature says that it is desirable to cache the IndexSearcher, > but there's no

Re: Caching IndexSearcher in a webapp [was: Find "latest" document (before a certain date)]

29 aug 2007 kl. 14.32 skrev Per Lindberg: For each search request (it's a webapp) I currently create a new IndexSearcher, new Filter and new Sort, call searcher.search(query, filter, sorter) and later searcher.close(). You really want to reuse the IndexSearcher until new data has been added t

Re: Caching of BitSets from filters and Query.equals()

Chris Hostetter wrote: : equals to get q1.equals(q2). The core Lucene Query implementations do override : equals() to satisfy that test, but some of the contrib Query implementations do : not override equals, so you would never see the same Query twice and caching : BitSets for those Query ins

Re: Caching of BitSets from filters and Query.equals()

: equals to get q1.equals(q2). The core Lucene Query implementations do override : equals() to satisfy that test, but some of the contrib Query implementations do : not override equals, so you would never see the same Query twice and caching : BitSets for those Query instances would be a waste

Re: Caching of BitSets from filters and Query.equals()

Chris Hostetter wrote: : I was hoping that Query.equals() would be defined so that equality would be : based on the results that Query generates for a given reader. if query1.equals(query2) then the results of query1 on an indexreader should be identical to the results of query2 on the same inde

Re: Caching of BitSets from filters and Query.equals()

: I was hoping that Query.equals() would be defined so that equality would be : based on the results that Query generates for a given reader. if query1.equals(query2) then the results of query1 on an indexreader should be identical to the results of query2 on the same indexreader ... but there in

Re: Caching of BitSets from filters and Query.equals()

On Mar 6, 2007, at 6:35 AM, Antony Bowesman wrote: Erik Hatcher wrote: Have a look at the CachingWrappingFilter: It caches filters by IndexReader instance. Doesn't that still have the same issu

Re: Caching of BitSets from filters and Query.equals()

Erik Hatcher wrote: Have a look at the CachingWrappingFilter: It caches filters by IndexReader instance. Doesn't that still have the same issue in terms of equality of conditions that created t

Re: Caching of BitSets from filters and Query.equals()

Have a look at the CachingWrappingFilter: It caches filters by IndexReader instance. Erik On Mar 6, 2007, at 2:03 AM, Antony Bowesman wrote: Not sure if I'm going about this the right way,

Re: Caching

On 2/14/07, Mark Miller <[EMAIL PROTECTED]> wrote: Not to get off topic, but I was curious Yonik, what does solr do if many updates come in at a time opening and closing a writer each update...does the first update kick off a warm operation, then before that warm is done the second updates kicks

Re: Caching

Not to get off topic, but I was curious Yonik, what does solr do if many updates come in at a time opening and closing a writer each update...does the first update kick off a warm operation, then before that warm is done the second updates kicks off a warm operation, and then before that warm i

Re: Caching

On 2/14/07, Kainth, Sachin <[EMAIL PROTECTED]> wrote: I have an index with 5.2 million records (each record containing 3 fields) and it sometimes takes about a minute and a half for results to come back. Doe to sort fields (and other factors), the first query can be slow. Solr has built-in supp

RE: Caching

slow and that perhaps I should implement a caching strategy. -Original Message- From: Erick Erickson [mailto:[EMAIL PROTECTED] Sent: 14 February 2007 14:11 To: java-user@lucene.apache.org Subject: Re: Caching This is really an unanswerable question, since, to steal a phrase, "It de

Re: Caching

14 feb 2007 kl. 14.57 skrev Kainth, Sachin: I have read that Lucene performs caching of search results so that if you perform the same search in succession the second result should be returned faster. What I wanted to ask is whether this caching is any good or whether it's a good idea to add s

Re: Caching

This is really an unanswerable question, since, to steal a phrase, "It depends" ... Do you have any reason to believe that the current performance is inadequate for you application? Caching is notoriously difficult to get right, so I wouldn't go there unless there is a *demonstrated* need. By dem

Re: caching lucene

On May 18, 2006, at 3:06 AM, Alberto Marquÿe9s wrote: Hello I need to know if Lucene has breaks for search, and in case of having it if I can form it and like becoming it. huh? I'm sorry, but I do not understand the question. Erik -

Re: Caching Results

Well, this depends, in case you have a small index just some million documents that it make no sense. But in case you have some hundred millions documents and may use distributed searching it makes a lot of sense. Just check ehcache.sf.net i found it very useful. HTH Stefan Am 29.11.2005 um 1